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Introduction 
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of the term - hard computing. This chapter defines this term. Another term сай 
hybrid computing has also been introduced in this chapter 
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ecd 


1.1 Hard Computing 


The term — hard computing was first coined by Prof. L.A. Zadeh of the University of 


1 bir... TC A Pn i | 141 -L. te 1 : : l 4 : {їй ан à " m Fam z 
California, USA, in 1996 |1], although it had been used to solve different problems for a 
long time 

T | ir |- ~ ~ + | z "m lcm ev a APIT i marae | „ пи JPA 
Let us see the steps to be followed to solve an engineering problem, which are: 
• 1 he variables related to an : ngineering problem are identined first and then classified 


Into two groups, namelv input or condaition variables (also known as antecedents) 


and output or action variables (also called consequents) 


e The input-output relationships are expressed in terms of mathematical (say differen- 


tial) equations. 


e Differential equations are then solved analytically or using numerical] methods (if 


required). 


e Control action is decided based on the solutions of these mathematical equations. 


The procedure stated above is nothing but the principle of hard computing. 


1.1.1 Features of Hard Computing 

Human beings have their natural quest for precision and as a result of which, they try to 
model a problem using the principle of mathematics. Thus, hard computing could establish 
itself long back, as a conventional method for solving engineering problems. It has the 


following features: 








SOFT COMPUTING 
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E , *j yutions. Thus, contro! 
e As it works based on pure mathematics, 1t may yield precise solu l 
actions will be accurate. 
a. ага eng. i athematically 
e It may be suitable for the problems, which are eas) to model m y and 
whose stability is highly predictable. 
Hard computing has been used to solve a variety of problems, two such problems are 
stated below. 
1. Stress analysis of a mechanical member subjected to some sort о! loading using a 


Finite Element Method (FEM). 


rivative (PID) controller to 


2. Determination of gain values of a Proportional Integral Der 


be used to control a process. 


It is important to note that hard computing can be used, if and only if the problem 


of the real-world problems are so 


can be formulated mathematically. Unfortunately, most 
complex and ill-defined that they cannot be modelled mathematically or the mathematical 
modelling becomes highly non-linear. Moreover, some sort of imprecisions and uncertainties 


are inherent to these problems. Thus, it may not always be possible to solve the complex 
real-world problems using the principle of hard computing. 


1.2 Soft Computing 


The term — soft computing was also introduced by Prof. Zadeh, in 1992 [2]. It is a 
collection of some biologically-inspired methodologies, such as Fuzzv I (EL) Ne і 2 
Network (NN), Genetic Algorithm (GA) and their different Ae Ame um tha еш 
FL, GA-NN, NN-FL, GA-FL-NN, in which precision is traded for t С БЫ namely GA- 
ease of implementation and a low cost solution. Fig 1 р ТЕ actability, robustness, 


l shows a зсћетан : TET 
: T - . + HUWS а schematic diagram di- 
т К со ксн се > diagram indi 
cating different members of soft computing family and their possible interact А Control 
Zr possib 'ractions. Contro 
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algorithms developed based on soft computing may be computationally tractable, robust 
and adaptive in nature. 
“у А \ T i, Је 
1.2.1 Features of Soft Computing 
Soft computing is an emerging field, which has the following features: 


e It does not require an extensive mathematical formulation of the problem. 


e It may not be able to yield so much precise solution as that obtained by the hard 
computing technique. 


e Different members of this family are able to perform various types of tasks. For exam- 
ple, Fuzzy Logic (FL) is a powerful tool for dealing with imprecision and uncertainty, 
Neural Network (NN) is a potential tool for learning and adaptation and Cenetic Al- 
gorithm (GA) is an Important tool for search and optimization. Each of these tools 
has its inherent merits and demerits (which are discussed in detail, in the subsequent 
chapters). In combined techniques (such as GA-FL. GA-NN, NN-FL, GA-FL-NN), 
either two or three constituent tools are coupled to get the advantages from both of 
them and remove their inherent limitations. Thus, in soft computing, the functions 
of the constituent members are complementary in nature and there is no competition 
among themselves. 


e Algorithm developed based on soft computing is generally found to be adaptive in 
nature. Thus. it can accommodate to the changes ог a dynamic environment, 


Soft computing is becoming more and more popular, nowadays. lt has been used by 
various researchers to develop the adaptive controllers. For example, several attempts 
have been made to design and develop an adaptive FL-controller or NN-controller for an 
intelligent and autonomous robot [3]. 

Most of the real-world problems are too complex to model mathematically. In such cases, 
hard computing will fail to provide with any solution and we will have to switch over to soft 
computing, in which precision is considered to be secondary and we are interested primarily 
in acceptable solutions. 


13 Hybrid Computing 


Hybrid computing is a combination of the conventional hard computing and emerging 
soft computing. Fig. 1.2 shows the schematic diagram of a hybrid.computing scheme. Both 
hard computing as well as soft computing have their inherent advantages and disadvantages. 
To get the advantages of both these techniques and eliminate their individual limitations, 
one may switch over to hybrid computing to solve a problem more efficiently. Thus, a 
part of the problem will be solved using hard computing and the remaining part can be 
tackled utilizing soft computing, if it so demands. Moreover, these two techniques may be 
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Figure 1.2: A schematic diagram showing the scheme of hybrid computing 


complementary to each other, while solving some complex real-world problems. However, 
the users of hard computing often do not like soft computing people, due to their inertia 
and fight among themselves 

Nowadays, hybrid computing has been utilized 
engineering problems in a more efficient way. Ovaska et al. |4] provides with a survey 


by various investigators to solve different 


on various applications of hybrid computing. Some of the possible applications of hybrid 
computing are: 


1. Optimal design of machine elements using Finite Element Method (FEM) and soft 
computing — Several attempts have been made to design different machine elements 
using the FEM. However, the quality of these solutions depends on the selection of the 
elements, their size and connectivity. Moreover, material properties of these members 
may not remain unchanged during their applications. Thus, there exists fuzziness in 
both the FEM analysis as well as material properties. This fuzziness can be modelled 
using soft computing, before the FEM analysis is hnally carried out to obtain the 
optimal design of machine elements [О]. 


2. PID controller trained by soft computing — A PID controller Is one of the 
used controllers, nowadays. Its gain values (Кр, Kr and Kp) are det 
ematically based on a fixed condition of the environment. Thus. 
change in the environment, the conventional PID controller | 
an optimal control action. То overcome “this difficulty. t} 
using the principle of soft computing. In a dynamic 
values of the PID controller can be determined on-] 
or NN-based expert system. 


most widely- 
ermined math- 
if there is a sudden 
шау not be able to yield 
e gain values can be tuned 
environment, appropriate gain 
Ine, using a previously tuned FL 


It is important to mention that the selection of a technique depends on the problem to 


generally first (ту to solve it 
reasons, We may switch over to soft 
ıy brid computing, if the problem so 


be solved. After gathering information of the problem, we 
using hard computing. However, if it fails for some 
computing. Sometimes, we should take the help of ] 
demands. 

This book deals with the princi 
computing is beyond its scope. 


ple of soft computing and a detailed discussion on hard 
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1.4 


Summary 


This chapter has been summarized as follows: 


1. 


1.5 


ә 


Hard computing works based on the principle of mathematics and it generally yields 


precise solutions. However, we can go for hard computing, if the problem is well- 
defined and easy to model mathematically. 


As most of the real-world problems are complex in nature and difficult to model 
mathematically, hard computing may not be suitable to solve such problems. One 


may switch over to soft computing to solve the above problems involving inherent 
imprecisions and uncertainties. 


Soft computing is a family composed of different members, namely FL, NN, СА and 


their different combinations, in which precision is traded for tractability, robustness 
and a low cost solution. It can provide with some feasible solutions for the complex 
_ 


| meis Freue Rap esee à 
real-world problems. 


The concept of hybrid computing that combines the hard computing with soft com- 


Li " " '* 
putting, nas also been emerged to gel 


advantages from both of them and eliminate 
their individual limitations. 


Selection of a suitable computing scheme for solving a problem depends on its nature. 


Exercise 


Define briefly the terms soft computing, hard computing and hybrid computing with 


some suitable examples 


How do vou select a suitable scheme 01 computing (either hard computing or soft 


i а де и Ln К ? 
computing or hybrid computing] to solve a particular problem 
а. ж uo Oe - M =. 
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Optimization and Some Traditional 
Methods 


mization and discusses some of its traditional 


/ | 
шеслов5 (also known as mventional or claecical ‘| | r 3c | ~ 
| 3 | m = ee veuveluonal or Classical methods). Traditional methods of Optl- 
well as direct search techniques. A detailed dis- 


CUSSION On ап these traditional methods of | ptimization is beyond the scope of this book. 
An attemnmnt ac Koar moana in thai | moles mus di; —X Rex т 1. tM E cz Lor к 

An attempt has | n made 1n this chapte | explain the rking principle of a few meth- 
Ik Method and Steepest 


ods only, namely Exhaustive Search Method. Random Wa 


Descent Method 


2.1 Introduction to ( 


(Optimization is the process of finding the best one, out of all feasible solutions) To realize 


the need тог ап отиттта топ. е US SUuDDOSst глас We ale EOLIE | aesign ch suitable Pear- 


CLIL МРАЗ А А 


d propeller shaft of a ship. We generally use 


1 1 1 7] + 1 Тал, 
роҳ to be used between a source ol power ап 


the traditional principle of machine design (which includes determination of stress-strain 
в a к ' | a n à iil ' Is ml | 3 aT. | "E ~ ч” РТ 
we use only this traditional principle, there is a possibility that the designed gear-box will 


ave no practical importance. at all. To overcome this difficulty, a designer should use an 
ја. = LX pi di LIC al ипи || 
optimization tool along with the traditional prin iple ol design. An optimization tool can 
make a design more efficient and cost effective 
The concept of optimization 15 defined mathematically as follows: Let us consider y to be 
the function of a single variable т, that 15, de J T) If the first derivative of this ee 
that is, f (x) becomes equal to zero, that is, f (т) = 0, at a point т = с", we say that 
CENTER | or inflection point exits at that point. 


either the optimum (minimum ог maximum) 
An inflection point (also known 
nor a minimum one. To further 
non-zero higher order derivative denoted by m. 
below. 


as a saddle point) is a point, that is neither à HIRED 
investigate the nature of the point, we determine the first 
Two different cases may arise as stated 
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2. If n is seen to be an even number, 2" is a loct ! b S 
for declaring it either a local minimum or а local maximum point, the following two 


conditions are checked: 
e If the value of the derivative is found to be positive, z* 1s a local minimum point. 


e If the value of the derivative is seen to be negative, z* is a local maximum point, 


A Numerical Example: 


z i 
Determine the minimum/maximum/inflection point of the function f(r) = > t == for the 
positive values of т. 
Solution: 
The function is: 
f(z) = = c 

Fig. 2.1 shows the plot of this function. Its first order derivative is given by f (2) = == um 

ou | 

G [ | 
15 = — = L a ك‎ ь n „| ipm] 
- ) 5 0 | R J 10 


Figure 2.1: Plot ој the function f(a) = z? 4 12 


By considering f (x) = 0 and solving it, we get z = 5.0 
Therefore, the minimum/maximum /inflection point Wines 
The second order derivative of this function is given um 
Акт = 5.0, f" (z) = 3.0 0.0. | 


Therefore, n is found to | | 
> M si С % ап | Ia b aeo 4 
"oko and it indicates that the loc 


at m 5.0. 
by f (ту= 1+ 250. 


al optimum exists at 
As f (т) is seen to be equal to 3,0 
minima exists at т = 9.0, 

The minimum value of f(x) 


(a positive number), it can be declared that the local | 


comes out to be equal to 37.5. 
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2.1.1 A Practical Example 


Let us suppose that we will have to design an optimal wooden pointer generally used by a 
speaker while delivering a lecture. The pointer should be as light in weight as possible, after 
ensuring the conditions that there is no mechanical breakage and deflection of pointing end 
is negligible. This task can simply be thought of selecting the best one, out of all pointers 
contained in a bin. | 

This problem can be posed as an optimization problem as explained below. Fig. 2.2 shows 
the pointer in 2-D. The geometry is specified by its diameter at the gripping end d (the 
diameter of the pointing end has been assumed to be equal to zero) and length L/. Its density 
p has been assumed to be a constant. [he geometrical parameters, namely d and L’ are 


— —_ — м 





y- 
О 
=A: 
ni 
Figure 2.2: A wooden pointer. 
known as design or decision variables and the fixed parameter p is called pre-assigned 





parameter. The design variables have some feasible bounds, which are known as geometric 


or side constraints. The aim of this optimization is to search a lighter design, for which its 
weight has been expressed with the help of an objective fünction. Moreover, the pointer 





should be able to satisfy a few constraints (in terms of its strength and deflect ion) during 


its usage, which are known as functional or behavior constraints 


The above optimization problem can be expressed mathematk ally as follows: 


T! 
"na L p 


Minimize Mass of the pointer M - Ee (2.1) 


р 


subject to 
deflect lon 0 = allowable 4 | A Ut Шы 


strength of the stick s > Srequired 


and 


qui < d < dum. | UN T, р 


[mn = I < [тат | 


Here, equation (2.1) is called the objective function. The constraints related to deflection 
and strength of the stick are known as the functional constraints. The lower and upper 
limits of d and L/ are expressed as the geometric or side constraints. 
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2.1.2 Classification of Optimization Problems 


кг i sed below. 
E NEST of ways as discus 
Optimization problems have been classified in a number j i 
imizatio: olved in the objective function and cog. 


| | ге of equations inv | i AC 
1. Depending on the nature of еді amely linear and non-linear optimiza. 
ame 


straints, they are divided into two groups, n 
tion problems, which are defined below. 


An optimization problem is called linear, if 


• Linear optimization problem are found to be ET 


) traints 
both the objective function as well as all the cons 


functions of design variables. 


xample: | | a 
Exar iple =f o ол шо (2.2) 
Maximize y (21,22) = 211 = 

subject to 
51 T Г? < j 
ЭТ} 209 < 10 
апа 
j Lo j (J. 

* Non-linear optimization problem - Ап PIE OE problem is known аз 
non-linear, if either the objective function or any one of the functional constraints 
Is non-linear function of design variables. Moreover, both the « )bjective function 
as well as all functional constraints may be non-linear functions of design vari- 
ables in a non-linear optimization problem. 

Example: 
Maximize у = f (T1, 29) = г“ +23 (2.3) 
subject to 
ri + т5 < 629 
ri + 25 < 133 
and 
01, То U 


2. Based on the existence of any f 


sified into two groups, such as un-constrained (without 
and constrained (w hen at least - one functional consti 
problems, the examples of Which are given below. 


unctional constraint, optimization problems are clas- 


any eee al constraint) 
raint is present) optimization 





* Un-constrained optimization problem 


Minimize y = f (2,29) = (ту = 5)? + (22 — 2)? (2.4) 


where 
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11,22 2 0. 


e Constrained optimization problem 


bo 
| 
to 
ai 
ы 


Minimize y = f(z,,22) = (тү — 5)? + (a: (2.5) 


subject to 


and 


[1,09 „> (), 


di 


3. Depending on the nature of design variables, optimize: ation problems are clustered 





into three groups, namely integer pr Ogr dung ге: al-valued programming | and 


mixed- integer programming problems, which are briefly discussed below. 


e Integer programming problem - Ап optimization problem is said to be an 


inte ger progran iming r problem, if all the design variables take only inte ger values. 
Example: 
Maximize y = } (21,29) = 271 + 72 (2.6) 


subject i 


ari d 


ч К ^ - |. x „== 
11,22 are the Integers. 


e Real-valued programming problem - An optimization problem is known as 
a real-valued r programming problem, if all the design variables are bound to take 
only real values. 


Example: 


© 
=] 
d 


Maximize у = f(zi, 22) = 211 + 72 (2. 


subject to 
т +29 < 3.2, 
921 + 2тә < 10.0 


апа 


I] ' L2 2 0.0, 
21,22 are the real variables. 
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o 


è " MX" amb T < 
. ТЕ is an optimization pl oblem, in 
variables take rea] 


i snmml yroblem о 
e Mixed-integer programming | кє and the remaining 


- . ате їпбере 
which some of the variables are 11066 
values. 


` fj | da 1 | l l Т 1 # 4: 3 ( 8) 
N 7 T Г چ‎ ай k j - I ' 4 ? i I ) 


subject to 


ar = 
ry L то + Та fouls 
v 41 
|| h 
Jy ML | Cn 
OL | 1 P inm. 
А i ~ 
апа 
i (iy 
| i Tc ] | ] n is ri real ма гјабје. 
[ of ET ILI L5 it | с 
1 L LI UEM 


2.1.3 Principle of Optimization 


lai inciple of optimizati let us consider a constrained optimization problem 
То explain the principle of optimization, let us consider a con: | 


as given below. 


Minimize у = f(ri, 25) ri — ај“ + (T; | 2.9] 


subject to 


where a and b are the constants 
Fig. 2.3 shows the plots of the constraints. Depending on the nature of these ci straints. 4 

feasible zone has been identified. A ny point lying in the feasible Z0ne is а B MR 2 : 
for the optimal solution. The points residing inside the feasible bons EE У ie : à CO an 
whereas the points lying on boundary of the feasible zone are кшй z ен SS pond 
Thus, an optimal solution c free point or и bound po = Pound points Po 

feasible zone. To determine the optimal solution. t] fs Ba ees peer m ihe 
different fixed values of y Оз, „5 Procedure is adopted. For 


ва j | * 1o ~" | | i | ље Р 

The point, at which one * Mica iM | л t | тх кошт plots of the objective function 

= и dete | "Чг plots just touches the feasible n- Bes bu 

optimal point and the corresponding optimi: “nes the feasible zone, 15 declared the 
ОПЕ Optimized function value is determined 


and m indicates the number Of f 


unctional constraints. 


an be either a 


је follow} 
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Side constraint 





| 1 سے‎ Й ы 
——————— —————— ASQ aan RR TT 
| ^ EM к Жыт кр r. 
1 ы ы a Рата ur Ж, 

k 








2.1.4 Duality Principle 


id lal function ol ial enot уу | which 1s 

Let us consider a unimodal tunctiol | le den 

а " ANE 1 ir bs ч 
be minimized. І пе problem 

Minimize 2.10 

subject to 
= - АЕ the Буги + the function reaches its 
| 1g. 2A shows the DIO! O1 11! ||| || | | | 


ini ling to a value ol T = 1 ibove minimization problem can 
minimum value. согтезропишу LO ch Vlll | | | ] M [ Ее] 


һе posed as a maximum probDieln de 


Џ 


Maximize f(a | (2.11) 
subject to | 
А > 0.0 
this conversion of minimization problem into а 
remain the same 


16 is important to mention that through | 
Inaximization problem, the value ol т at which the optimum occurs will 
as r' It is known as the duality principle of optimization. 
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A f(x) 





e of optimization 
е) ; lat used to explain the duality principle of ој 
Figure 2.4: A function ри ux ИО БА 


2.2 "Traditional Methods of Optimization 


‚ ‘aditional methods of optimization. Various methods are 
А huge literature is avallable on traditional m ПАРАДЕ 


ы 


LUIE е је problem-dependent. For 
| : > 5$ ‚1 ‚ tio af on appropriate One 15 problem AE pen је L. 
in use (refer to Fig. 2 | and the selection oi an Ж. : 


[ 


- + 1 й F " | 3 + T 1» pa = 
ог solving the linear programming problems, whereas 


exalnpie, we use a gTOUD OI Mit | 


e i 1 Г» r ry li а] on 

for non-linear problems, we take the help of some other methods. In a non-linear opti 
1 LOGS LHCE MOAGI L LP. Li | | 

> | pl iT iT i { 1 ° à уе riz e ОГ a 

mization problem, the objective Iunction may De Ci imposed Ol е he I d single Vall ib] JI a 


number of variables. A single variable non-linear optimization problem can be solved E 
analytical, numerical methods. The analytical method works based on the principle of dif- 
ferential calculus. Numerical methods include both the elimination as well as interpolation 
methods Multi Variable optimization problems шау ре either constrained ОГ unconstrained 


in nature. Unconstrained optimization problems can be tackled utilizing either direct search 
or gradient-based methods. On tl 


he other hand, there are some direct and indirect methods 
to solve the constrained optimization problems 


A detailed discussion on all these tradit ional optimization methods is Devond the scope of 


books [6, 7] for the above purpose. Although 
thods are explained below, in detail. 


this book. Interested readers may refer to the 
the scope is limited, a few traditional me 


2.2.1 Exhaustive Search Method 
<< С0 ме поа 


Let us consider a unimodal function y (that is. it 


| : has only one peak) of a 
defined in the range of (rn, ymaz) where zin and gaz 


of the variable т, respectively. Our 
range. The proble 


single variable 1. 
are the lower and upper limits 


function y — /(т) in the above 
as follows: 


aim is to maximize the 
m may be mathematically stated 


Maximize у = f(z) (2.12) 
subject to 


an < т< рта 
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Аа - at سے‎ —HÓá— = 
| 
А 1 | 
Е prm? pP reel Fi 16 iOS 
E EBLE 4 Шет ud = 
п ке 1 | 
| таят nre rmm | 
ерга tear |. Quadratic тисгроаноп ше 
i, WEST CCU зш ДУ 
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\ „РА кєй» sam 2. Cu nterpolal 
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3. Lhchotomous Seat 
a Lee ee mair i 
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j ! Conjugate gradient meurod 2. Cutting plane method method 
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3. Pattern search 
4 Rosenbrock' s method 


mau 
Ls 
E 
- ' 

= 
~ 

4 
i 


Figure Z0. Classification of traditional methods ol optimization. 
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12131421577 | 
= 
T yp 4 | | . A +l " e of an exhaustive search method. 
Figure 2.6: A schematic Hagram usei ы: aoni Vr dep | 
Fig 2.6 shows спе plot of the function i De optimizei јРрепаш On LIC аезпеа accuracy 


in the solution, let us suppose that th inge of the v Je, that is, | —Е "Jis divided 
into m equal parts. Thus, the number of equi-spaced intermediate points lying in this range 
becomes equal to (т 1j. LHe small change in т, that is. A given by the expression 
[7a гїї! САЖ =" 

=, The following steps are used 1 


ri 





the function: 
e Step 1: We set 


e Step 2: We calcul 


E ! EI A i 021), J (72), f (a ‚) and check fo 
the maximum point І neces 


If / (ж, Шз || Г > fla 


i LAILI 11 bia) || 1165 1П - ће ‘AT OA а VPN У ri 
terminate the program. ЈЕ ( » "1 We 


Else 


| "| тур ] + | 


LD L3 pre Uto © | 
L3 L2 (pre У СЕ [ } | Ar 
Й Step 3: We check whether +. exceeds paa 
If z4 does not exceed тта wa o SES 
ues 4 БО to Step 2 
"ISe we say that | 


the maximum does not lie in 


the range of (рит 


лат J. 
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A Numerical Example: 


(2.13 


Maximize y = f(x) = 20° =a 
subject to 
i} GO Jt 
Solution 
Fig. 2.7 shows the plot of function 
G 1^ el | і | 1 | 
| 
e Step 1 Is T into n = 10 equ 
rts. V iat 1s, Аг as follows: 
We Si 
0.0 
p г 7] || [] aA. || 2 
|| F- Ti | cor - = - 
Ta ra Ат 0.2 4- 0:2 ).4 
e Step 2: We determine the function values at т = rj, 7 and тз like the following. 


0.0 
0.072 


0.256 


|| (ЈА) ) 
[(0.2) 
- f (0.4) 


/\т\) 
| 9 } 


IU 


f (тз) 


Therefore. 


' 


| 


al 
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f(z1) < (202) < jJ (x3) 
A s 
Thee ska maximum does not lie in the interval (0.0, 0.4). 
Е IUS. HC Liat FP 
$ 
“ е TY". .. 
e Step 3: We set 
] = Drevious = (2 
ту = previous) = 0.4 
А А | ) ne 
Та = esent) + A3 10.2 = 0.6 
|| step | UA | 1 ti - tinctim а || Donc | ИЄ 
Г J | | ر‎ 
\ | 
| i 
Ci Ж 
ы " r 
• Step 5: Wes 
і 
2| pri A · | L 
• Step 6: We 
| ЈА 1 C | r] 
Lig | i 
H | ү 
é + 
| 2) ] (0.6 | E | 
} 
JJ L6] X 0.765 
l'herefore 
ri 7 r 
= " | Г | | | | 
4l | ad J [| ! + | 
на = ар | 
Рту FE. 
HLS. LILE mawr кы : | 
{Hah does not lie in t] | 
| п the interval (( 4 + 
val (0.4, 0.8) 
1 А" + Р +". У 
• Step 7: We зе ), 
= 


ү" : j 

a | -— 12 ргеуј 2 е - 
A i JUS | em | ' 

12 Кб 


%3 (previous) = 

a B 4 > | 0. 

T2(present) + Ar — 0.8 j | 2 | 
"PT U2 = 1.0 


L3 - 
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e Step 8: We calculate the function values as follows: 


f(z1) = f(0.6) = 0.504 
f(z2) = f(0.8) = 0.768 
јаз) = f(1.0) = 1.0 


Therefore, 
f (21) < f(z2) < f (x3). 


Thus, the maximum does not lie in the interval (0.6, 1.0). 


e Step 9: We set 


ху = то(ргех1оџ5) = 0.8 
то = 23(ргеуіоиѕ) = 1.0 
тз = r2(present) 1 Ат = 1.0 + 0.2 = 1.2 


5n Cho follows: 


e Step 10: We calculate the function values as 


Therefore, 


loes not lie in the interval (0.5, 1.2). 


Thus, the maximum does 


e Step 11: We set 

to(previous) = 1.0 
(previous) = 1.2 

тт ese P -Ат=12+02=714 


T3 = 


e Step 12: We calculate the function values as follows: 
fíiz1) = аак 
f(2) = 
јаз) = 


T: 


Therefore, 
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ee 


f (x1) < f(z2) < f(z3). 


| t | l. J. 
lta 3 1 n inte val (1 О, | 
I ك‎ } i iximum does not lic IIl the I 4 
4 hus, the ша. 


2. е 56! 
e Step 13; Wes | ; 
T] I9 (previous) шй. 
То та(ргем1015) | 


єл 6 
- ro(present) + Ат = 1.4 + 0 1.6 


+ died 


| = 
5 n^ { low 2 


~ e calculate the function value 
e Step 14: We calcula 


| J f | 2 | 152 
f (a f(1.4) = 1.176 
"| = : т i 
f f | D 1 (24 
Therefor 
| Í 
|| iS. ЕД L = ! 
| is obtained using n = 10. However, a finer interval 
: IT T ATCO ITET | Ol ] (па! |= E | f 15 | LITIPTC] ] ] | 
J Li РО h | | 
“a и тез f à! E. i | те 4 үтү | | De equa |, | | 85 analvticallv апа 
Note: The maximum value о us function is foun be equal | 3 \ 
i F pees 
$ | [ Jatt 


' т " к || 
Random мак method Ls Г Н Пи direct search methods IT] whic h the с 


earch is carried 
it its derivative informat lon 1s not utilized [6]. Here, 


determined using the previous solution (that 15, 


(2.14) 


в m а Hp; уљу — 2 1S the number of variables, A indicates the step length, u; is 
a unit random vector determined Ef g , 


ri. Lp2)5 эп) , Where ny opo Tn) 
n As ' | Y „2 2 244 j 
.Ü and 1.0. such that а БОИ гүз = КА < 1.0. 


"termine the optimum (Say minimum) solution: 


аз ш = пе Á (T1 Te r | 


are the random numbers lying between E 
The following Steps are considered to d 


* Step 1: We start Ww 


| th an initial solution X}, created at random. We se 
to A, € (permissible ' SE 


ес minimum value of А) and maximum numbe 
ыле [ог ИП ovement of 2 80 LIO IS | | 
[ f the solution (that IS, N). We determin 


Д = f (X). 


t initial values 
r of iterations to be 
e the function value 
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e Step 2: A set of n random numbers (lying between —1.0 to 1.0) are generated and 


we calculate ш. 


e Step 3: We determine the function value using the expression given below. 


fo == [(Х») —- ЈА + Ац) 


e Step 4: If f < fi, then we set X1 = Xi + Аш, Л = Ja, and repeat the Steps 2 


through 4. 


Else repeat Steps 2 through 4, up to the maximum number of iterations №. 


e Step 5: If a better point X;,; is not obtained after running the program for N 


iterations, we reduce А to 0.54. 


e Step 6: Is the modified (new) А < є! 
If no, go to Step 2, 


Else we declare X* = X,, f* = fı, and terminate the program. 


A Numerical Example: 
Minimize (21,120) = 421 


subject to 


using random walk method. 

Assume: A = 1.2, є = 0.075, N = 10) 
Solution: 

Fig. 2.8 shows the plot of the function. 


Some of the iterations carried out to solve the 


method are: 


above problem using the random walk 


e Iteration 1 It consists of the following steps 


— Step 1: Let us consider an initial solution generated at random as given below. 


X == 


Sogi 


0.0 


We determine the function value Ji corresponding to X; like the following. 


fi = Ху = 0.0. 


— Step 2: Let us also assume that the following two random numbers lying between 
—1.0 and 1.0 have been generated: 








fix 1,х 4l 


1400 


1200 
o00 
Bu 
& uu 





Figure 2.8: Plot of the function f(zi, 22) = 421 + 325 — 6271272 — 411. 


We calculate R (ri +175)? 0.5385, which is less than 1.0. So, we consider 
the above Г values [ог determination ої the unit random Vector as given below. 


ШОО пл |__- ј 05 | _ | OS 
epee (ro | 993985] 0.2 J | 0.3714 


E Step 3: We deteruiine the function value fo AS follows: 
f2 = f(X2) = f(Xi + Аш) = f(—1.1142, 0.4457) = 12.9981. 


—1.1142 
$^ а Hs ПО [ 8. КЕ "Uf AT “Га i 4 4 Р - 
0.4457 cannot be considered as Xo and we go for the next 


Ав fo > ћ, 
iteration. 


e Iteration 2 


k E x. | 00 "us 
Step 1: We have X1 == 4 0.0 and fi — F(X = 0.0. 


— Step 2: Let us consider another set of random numt 
1.0 as follows: 


ers lying between — 1.0 and 
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ашынын 


(n| [9000 | 
| Г? | | 0.1 | 


We calculate R = | гі + rj)2 = 0.1000, which is less than 1.0. So, the above set 


F = xxl | | i TM | | od 
X r values can be used to determine the unit random vector like the following, 


EEA J 0.001 | Í 0.01 | 
com щч щн КО TM 


Step 3: We determine the function value fz as follows 


f(X: TE See а f (0.012. 1.2) = 4.1862. 


1 гы: Ј2 x h ^ = i will not De considers d а= AQ and Wwe Bo for the next 


E As cannot be obtained alter running the program 101 № 100 iterations, we Set Anew = 


() J) K || p Р >> f We TPH || Ln Грета! [1 i і xplained shove. else We declare optimal 


ndm — == 


2.2.3 Steepest Descent М. thod 


Steepest descent method is one of the gradient-based methods, in which the search direc- 
Gon is dependent on the gradient of the objective function to be optimized [6]. Thus, this 
method is not applicable to a discontinuous function. Let us deine the term — gradient of 


г п, р pla 1 | (поа 
Gradient of a function: 
Let us consider a continuous function oi тп Variables аз piven below 

ERR ETC (2.16) 
Gradient of this function is defined as the collection o partial derivatives with respect 
da cau lt Ll , ~ а ти - | 
to each of the тп variables and it 15 denoted as follows 


EH —, — ne ) (2.17) 





It is important to mention that the rate of change of a function is found to be the maximum 
along its gradient direction. Thus, the direction ot its gradient 15 nothing but the direction 
of steepest ascent. In a steepest! descent method, the search is carried out along a direction 
Opposite to its gradient. 
Principle of the method: | e: 
We start with an initial solution ЛХ], created at random and move along the search direction, 
"according to the rule given below until it reaches the optimum point. 


(2.18) 


a Хы = Xi + А 5, 








ў 


SOFT COMPUTING 





-th iterations, respectively, 


where X; and X,,; are the solution vectors at i-th and (i + 1) 


ction 6: = — V fi 


AF represents the optimal step length along the search dire 
ermination Criteria: 

Any one of the following two crit 

algorithm. 


eria can be considered as the termination criterion of the 


1. If the rate of change of function value, that is, SN 
је algorithm 18 terminated. 


"ЊЕ 


IO) Pu becomes less than or 
equal to a pre-defined small quantity €;, tl 

2. If the derivatives 5L] j = 1,2,...,m, come out to be less than or equal to a pre 

, "О |? ма, 


defined small quantity єз, the program is terminated. 


Advantages: 
It is one of the most popular traditional methods of optimization, due to the following 


reasons: 
|. It has a faster convergence rate. 
2. The algorithm is simple and easy to understand and implement. 


Limitation of the algorithm: 


The search direction of this algorithm is nothing but the steepest descent direction, which 
15 Opposite to the gradient direction. As gradient is a local property of the function (that 
means, it may vary from point to point lying on the пе! топ), there 15 a chance of the 
solutions of this algorithm for being trapped into the local minima. 


A Numerical Example: 
Minimize f(ri.r5) = Ar“ + 37° Gr, + 7T ? 
J V1, v2 UL] T oTa — 02122 — 4T] (2.19] 
subject to 


= l Ü. < Ti To < 10.0. 


| 


Fig. 2.8 shows the plot of the function. A few iterations of the steepest descent method 
used to solve this problem are: ا‎ MURS 


e Iteration 1: 
Let us assume the initial solution created at random as Хү = 0.0 


" ү! 0. 
The function value at X, is determined as Ii = 0.0: Я 


Gradient of the function is obtained as follows: 


д Д 
gf- 5 _ ) 62%) —6ш0—4 
z3 J =T + бло 
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Now, the gradient of the objective function is found to be like the following at X4. 


— 4 


Vh=Vi(Ai)= 4 o 


Therefore, the search direction at X, is determined as follows: 


1 J | - | 
oy = V f | 


To determine X», we need to know the value of optimal step length, that is, Aj, for 
the estimation of which, we minimize 


F(X, + Ay x 51) = f(4A,,0) = 64 M — 16A1 


' ‚ di ; | ‚ 1 ; ^ Ма 
We determine г and put IL equal to UU | hi optimal Step length is found LO be 


ПА | 


equal to ў that is, Аз т. 


We calculate Xû = X + A154 =< 2 | 


The function value ai X» is calculated as 12 L.U 
We determine the gradient of the function at Xo, that is, 


Therefore, Хо is not the desired optimum point and we should go for the next iteration. 


Iteration 2: 
ы и i | , de rut тег тү 4 | Ua: 
The search direction at Хо is determined as 1000%5: 


To determine X3, we need to know the value of optimal step length, that is, Ад, for 


~" 1 


the estimation of which, we minimize 


“одише 


= 
=“ 
Bo = 


f X3 + Ag x 52) = f(g, 3A2) = 27 


| а m ert at је C 
We put ES = 0.0 and get the optimal step length, that is, Aj = б. 


BI ta] — 


T4 F A # af —^ 
We calculate Хз = X2 + 4252 7 


T 


The function value at Хз is determined as fy ==}. 
The gradient of the function at X is determined as follows: 
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З () 


v fs = VF (Xa) · | 0 


fa ain go for the next iteratie 

Therefore, X is not the desired optimum point. We should g lon. 
‚ à + 2 d i Та, the (0 uA 

| e /parried out, until it reaches рш 

» similar way, some more iterations аге to be carried | | 

In the similar way, some more 1t i found toba Me ЧАЙ 


~ ^E 1 is f ction 18 
(minimum) point. Finally, the optimal solution o! this functi 


ки = [20 | m th 


2.2.4 Drawbacks of Traditional Optimization Methods 


t E а 1 - ч " : : 4 dris uri lec 
Iraditional methods of optimization have the following drawback 


ing a traditional method de 


|| 
i 


: ~ ў ч ‘ : Г 4 i ۳ Р E dati С р | i | с: 
pends on the randomly chosen initial solution. If the initial solution lies in th« local 


at local optimum. Thus, if the user is lucky 


) 1. The final solution of an optimization problem solved 
basin, the final solution will get stuck i 


enough to choose the initial solution lying in the global n, the obtained optimal 
solution may be global in nature (refer to Fig. 2.9) 
P 
КА) 





Figure 2.9: А schematic diagram showing the local and global basins of an objective functi 
f a " if || B 3 Е , B | 1С ОП. 


2. For a discontinuous objective functi | 
i ү | je Jective псі ТОП, the Prax 3 ~ ige | А м 1 
gradient cannot be determined at the point 


of discontinuity. ;. the отада e. 
uty, Thus, the gradient-based methods cannot be used for such functions 


3, There is a chance of the solutions of 


а gradient-bas орта con .: Ж 
trapped into the local minima. -< Optimization method for being 


4. Discrete (integer) variables are difficult to h 
optimization, although there exists а separat 


andle using the traditional methods of 
e integer programming method. 








5. These methods may not be suitable for parallel computing. 


6. A particular traditional method of optimization may not be suitable to solve а variety 
of problems. 


Thus, it 15 necessary LO develop an optimization met пой, which is not only efficient but also 
robust in nature. 


2.3 Summary 
The content of this chapter may be summarized as follows: 


B = " i Р Р а евра a ~ | + ] 
I. А bri је! int roduc tion is given to the conce pi OI optimization. Different terms related 
to optimization, namelv decision variable, objective function, functional constraint, 


x: = 1 || ] з, zit - 
geometric constraint, and others, have been defined with the help of a pra ictical ex 
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The d draw vbacl ks of the traditional optimization metnoads nave реси ш gh ligh ited at the 
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end Ul bili chapter 


2.4 Exercise 


Define the following terms related to optimization by taking a suitable example: (i) 
decision variables, (ii) objective function, (iii) functional constraints, (iv) geometric 


p+ 


constraints. 


Explain the principle of optimization with a suit able example. 


to 


3. In a constrained optimization, an opt imal point is either a free point or а bound point 
lying in the feasible zone - justify the statement 


4 Formulate the optimization problern given in equation (2.9) as an un-constrained one. 
Identify its optimal solution in Fig. 2.3. 


| catre an pe 
5. Why do the solutions of a steepest descent method get stuck at the local minima ‘ 









, Minimize y = f(x) 
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f traditional methods of optimization. 


Discuss briefly the drawbacks о 


32 + х in the range of 0.0 < z < 10.0, Use (i) analytical 
MI. search method. | 


approach based on « liffer entis ] calculus and (ii) 


Minimize (11,22) = 4r? + 25 — 32122 + 021 1275 in the range of —10.0 € 21,23 $ 
| a E UA - 0.0 4 
10.0. Take the initial solution X1 = | 00 

(i) Use Random Walk Method. Assume step length À — 1.0, permissible minimum 

value of A, that is, € = 0.25 and maximum number of iterations № = 90. 

(ii) Use Steepest Descent Method. 
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Introduction to Genetic 
Algorithms 


Genetic and evolutionary algorithms, which work based on Darwin's principle of natural 
selection (that is, survival of the fittest), have been used as optimization tools. These 
algorithms include Genetic Algorithm (GA) [8], Genetic Programming (GP) [9], Evolution 
Strategies (ES) [10], Evolutionary Programming (ЕР) |11, 12]. А detailed discussion on 
each of these algorithms is beyond the scope of this book and it concentrates only on the 
GA. An introduction is given to a Genetic Algorithm (GA), one of the most popular non- 
traditional methods of optimization, in this chapter Several versions of GA are available 
in the literature, such as binary-coded GA [13], real-coded GA [14, 15], micro-GA [16], 
messy-GA [17], and others. Ап attempt will be made to explain the working principle of a 
binary-coded GA, in this chapte 


3.1 Working Cycle of a Genetic Algorithm 






,opulation-based probabilistic search and optimization tech- 









orithm (GA) 1s à 
based On the mechanism ol natural renetics and 


Genetic Alg 
nique, which works Jarwin’s principle 
of natural selection. The GA was introduced by Prof. John Holland of the Jniversity of 
Michigan, Ann Arbor, USA. in 1965, although his seminal book was published in 1975 [8]. 
This book could lay the foundation of the GAs. It is basically an iterative search technique 
working based on the concept of probability. The working principle of a GA has been 


explained briefly as follows (refer to Fig. 3.1): 









e А СА starts with a population of initial solutions generated at random. 


e The fitness/goodness value (that is, objective function value in case of a maximization 
problem) of each solution in the population is calculated. It is important to mention 
that a GA is generally designed to solve a maximization problem, Thus, a mini- 
mizatión problem has to be converted into a corresponding maximization problem às 





5 discussed below. 
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Figure 3.1: A schematic diagram showing the wot king cycle of a GA. 


Let us suppose that a function f(x) is to be minimized. It may be treated as а 


maximization problem as follows: 


Either Maximize — f(z), using duality principle, 
1 Т 


ог Maximize ү: 101-7 (ж) 55” 0, 


or Maximize Û fi fes son 
ы 1+7! Г | 4 Ї( M f | Mi | A U, 

or Maximize —— er and sa 

JI M LX ITILIZe 14 Lf (z))? > апа SO On. 


=25 Роршайоп of solutions is then modified using different operators namely re 
| pro uction, crossover, mutation, and others. The Fols SET БЕ 7 у s : 


e All the solutions in a population may Hot Ђелестајће saa: 

ues. An operator named ranr PA ced equally good in terms of their fitness val 
their dines values Thus ы auction utilized to select the good solutions using 
Hic ТЕ ае A "s à mating poo] consisting of good solutions proba 
E ERU: E. | с е у lat the mating pool mày contain multiple copies 
Eu solitis s у, | је size of the mating pool is kept equal to that of the 
BE ЖНА : consi. ered before reproduction. Thus, th fi of 

d expected to be higher than that of the ое е 


tion of solutions. : us 
feet on: | үрен exists a number of reproduction schemes in th | i 
n (such as Roulette-Whee] лез in the G A-literature, 


tion, ranking selection, and others [1 8) selection), tournament sele | 
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binary string is decided based on a desired accuracy in the value 
— — example, if we need an accuracy level of є in the va 


i a 


‚ The mating pairs (also known аз the parents) are selected nt random from the above 
pool, which may participate in crossover depending on the value of crossover prob. 
ability. In crossover, there is an exchange of pr 
+ iV. J 





erties between the parents and ля a 
сш of which, new children solutions are created. It is important to mention that, if 
the parents are good, the children are expected to be good. Various types of crossover 
operators are available in the GA-literature, such as single-point crossover, two-point 
crossover, multi-point crossover, uniform crossover, and others [19]. 


„ In biology, the word - mutation means a sudden change of parameter. For example, 
the crows are generally black in colour. However, if we could see a white crow by 
chance, it might have happened due to à sudden change in parameter on the gene 
level, which is known as biological mutation. In a GA-search, it 1s used for achieving 
а local change around the current solution. Thus, if а solution gets stuck at the local 


e 
nd ul 


minimum, this operator шау help it to come out of this situation a 
it may also jump into the global basin. ___ 


جح ے 


sequently, 








ра 


e Aíter the reproduction, crossover and mutation аге applied to the whole population 
of solutions, one generation of a GA is completed. Different criteria may be used 
to terminate the program, such as the maximum number of generations, a desired 


accuracy in the solution, and others. \ 


ry-Coded GA 


тис 


3.2 





Let us consider an optimization problem to explain the working principle of a binary-coded 
GA, as given below. 


Maximize y = f (11,72) (3.1) 
subject to 
min e m, < qm 
L1 ~] | 
pm < 19 Abe 


where x, and го are the real variables. The following steps are utilized to solve the above 


problem using a binary-coded GÀ: 


• Step 1 - Generation of a population of solutions: An initial population of 
solutions of size N (say N = 100) 200... depending on the complexity of the problem) 
is selected at random. In this type of GA, the solutions are represented їп the S a 
bin tri | A binary string can be comps | 104 
binary strings composed or سے‎ 7, E = s value, The length of a 
chromosome and each | ıe string is nothing but a gene vasue: ists 

: | a of the variables. For 

we will have 



















lues of a variable £1; 
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Сим 
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Bil Јах ` sidered to be prop 
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тат gu (32) 
4› | M... 
| == logy(- | ) 

| "а binary-code 
putational complexity of a bin и y 
e around. L log [,, | | 
each variable. Thus, in this 


| Let us also suppose that the initial 
o(-.bits long. 4! ~ | 


below. 


1 GA is dependent on 
It is obvious that com 


AS ee дек 
its string length L, which is found to ! 


| clone ; represent 
Let us assume that 10 hits are assigned to re] 


problem, the GA-string is only | | 
МО ي‎ А" "nido is: 
population о! CA-suings created at rando 


iw | 
О Д1] U 
Шу ae |) 
D 0 1 | 
Р | 
: | B | : " л Fî ness wu 116 01 РАД |] solution (that 1S. 
Step 2 - Fitness evaluation: Го determine the ntn Value oi 


- gi зз he calculated first. Knowing 
a : | ^ 3 |l Ts 1 r j "a АТ LO BT бс! ا‎ LE L LIL OL Ь ы ^F 
string), the real values of the variables - T] and тд al 


1 TG Ee n «eri | [enr T and decoded Value OI оде 
the minimum and maximum limits of a variable (Say 11) alc ! | | | 

| | ha rintnr | эЛ 1181 ү 
binary sub-string assigned to represent 21, 105 real value can be determined using Ше 


linear mapping rule given below 


ТЕТ È s L1 ут * | 
ry T | | | - = Ји [). | 9:91 


2° — ] 
where l is the length of the sub-string used to represent the variable r; and D rep: 
resents the decoded value of the binary sub-string. The decoded value of a binary 
number is calculated as follows: Let us consider a binary number 1 0 1 1 0. Its de 


| 
ч 


coded value is determined as 1 x 2* +0 х 2° +1 х 22 +1 x 21 + 0 x 20 = 29. The real 


value of second variable r9 is determined following the same procedure. Knowing the 


real values of the variables — тү and 22, the function value f (21,29) сап be calculated. 
that is considered as the fitness value of the binary-string. This proced 


to 


| | | ure is repeated 
determine fitness of each string of the GA-population. 


e Step 3 - Reproduction: АП the GA-strings сопќаіпе 


b i pus 'd іп the population may not 
e equally good in terms of their fitness values calculated in 


operator named reproduction is used to select the 
strings based on their fitness information. Sev 
developed by various i iga ле of | 

! ped by us investigators, Some of these are explained below. 


1. Proportionate Selection/ Roulette- 8 


Wheel £ ‘nis NEE 
scheme, the probability of a Tee el Selection - In th 


| for being selected in 
ortional to its fitness. It ig impleme 





2: nted with the help of f 


р 4 
m^ 
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Step 2. In this step, a2 
Bood ones from the-population of 
eral reproduction schemes have been 


the mating pool is CON 


ex pression given | 
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\A -Pointer 


N N times 






pL T | r М А ur - 
Figure 3.2: A Roulette-Wheel used to implement proportionate selection scheme. 


Roulette-Wheel shown in Fig. 3.2. The top surface area of the wheel is divided 
into N parts (where N is the population size), in proportion to the functional 
values — fi, f2,..., fn. The wheel is rotated in a particular direction (either 
clockwise or anti-clockwise) and a fixed pointer is used to indicate the winning 
area, after it stops. À particular sub-area representing a GA-solution is selected 
to be winner probabilistically and the probability that i—th area will be declared 
so, is given by the following expression: 

D жр. (3.4) 
The wheel is rotated /spun for N times and each time, only one area 15 identified 


ү Р r E TIL. = d "n ar chow Lx w in etail 
by the pointer to be the winner. [he procedure is shown below in detail. 
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In this scheme, a good string may be selected tor a number of times. Thus, the 
resulting mating pool (of size N) may look as follows; 
scel 
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Л и: | 
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population diversity and selection 
loration and exploitation, 
These two factors are ју. 


A QA-search depends on the factors like 


enm Ба: бай [ exp 
pressure, which are similar to the concepts of ех] 
| lv, as used in some of the Q A-literatures. 


e sense that if the selection pressure Increases 


respective 
versely related to each other in th 
2 " 11 | МЕ £A ER E a С" 'OT SEL 
the population diversity will decrease and vice-versi ill be focused onl 
+ pino AF | i | 
tion that if the selection pressure Increases, (he search wi а | \ y on 
| У ~ Lf | СА кај [ which, the diversi 
the good individuals (in terms ol fitness) and as a result ol Y " i Inti ш, 
g ' " TM " T " i g Ts 
will be lost. It may also lead to a premature convergene' ol 9 је solution to q 
her hand, a lower value of selection pressure тај 


| dex E ‘aneequently, a stagnation Ir 
not be able to drive the search properly and consequi ntly, a stagnation may 


Thus, selection pressure plays an important role in the GA-search and 
ry good search. It. T 


[t is important to mier 


sub-optimal value. On the ot 


оссо 
а proper amount of which is to be determined to ensure 
also to be noted that the fitness-based above repr duction scheme may lead to 


"m 1 у J T = ГЕ | j у тт 
the difficulties, such as premature convergence and stagnation [91] Го overcome 


these difficulties by providing with a better control over the selection pressure, а 
rank-based reproduction scheme was proposed by Baker |2] |; which is explained 


| © n Ww. 


Ranking Selection - Let us assume that there are only four binary strings in a 


population of GA-solution, whose fitness values are indicated by fi. fo, fa and 
Ја Let us also suppose that fi, fo, fa and fq take the values of 80%, 10%. 7% 


v4 


and 9/0 of the total fitness value (that is, $^? , f;), respectively (refer to Fig. 


33) Ша fitness-based proportionate selection is carried out. the probabilities of 





m Y = ry ғ). , 0 
Е Igure 3.4: А schematic diagram shi wine the hitness | ү : 
= E AU IIIS MISC pry OT | lonate selection 


De m tor 4 fa, fs and f4 are coming out to be equal to 0.8, 0.1. 0.07 
and UUS, respectively. Thus ha the e бид “Oy VS, М.Ш 
ono. ЕЕ ا‎ | за chance of occurrence of the premature 
Be BGs whe, ог је pro»'em, а rank-based reproduction scheme me 
Y E ple is discussed below. may 
The process of ranking selection consists of tw 


are arranged in an ascending order afik oo", 
© iscending order о! their fitness values (that i fa f fo, fi) 
iie C i Г H j 5, 4, 1 ғ | 
) ! 4903 423 J17* 


The string having the low - 

16 | Fag Г ъс + à 

qun dE 5 Owest fitness is assigned rank 1 and 

ranked accordingly. Thus, in thi | and other strings are 


o steps. In the first step, the strings 


РЫН - example, the strings fy, fs, f» and f, will be 
| cond step, a proportionate 


espectively. In the so 
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selection scheme based on the assi 
The percent (95) 


gned rank is adopted, as discussed below. 
area to be occupie ж RET T КЕТ ; ne 
the expression Z x 100. у} n by à particular string (i — th) is given by 
| er | where r; indicates the rank of i — th string. Thus, the 
strings fi, fo, Ја and Ја will occupy 40%, 30%. 2 


096 and 10% of the total area, 
3.4. It is important to mention that a rank-based 


T | | rO, = 
respectively, as shown in Fig. 





| Ig ul E AE А schematic diagram showing tI P rank-based Dri 


proportionate selection scheme is expected to perform better than the fitness- 
based ргорог! lonate seal ey L1 it 


3. Tournament Selection - ‘Tournament selection was studied first in Brindle’s 


dissertation |22|. In this scheme, we select the tournament size n (say 2 or 3) at 
random, which is a smaller number compared to the population size N. We pick 
n strings from the population, at random and determine the best one in terms 
of fitness value. The best string is copied into the mating pool and then all n 
strings are returned to the population. Thus, in this scheme, only one string is 
selected per tournament and N tournaments аге to be played to make the size of 
mating pool equals to N. Here, there is a chance tor a good string to be copied 
in the mating pool more than once. It is found to be computationally faster than 
both the fitness-based as well as rank-based proportionate selection schemes. 
Interested readers may refer to the work of Goldberg and Deb |18] for a compar- 
ative analysis of different selection schemes 

4. Elitism - This scheme was proposed by Kenneth De Jong [23], in which an elite 
string (in terms of fitness) 15 identified first in à population of strings. It is then 
directly copied into the next generation to ensure its presence. This is done 
because the already found best string may be lost, if it is not selected during 
reproduction using any one of the above schemes. 


• Step 4 - Crossover: In crossover, there is an exchange of properties between two 
parents and as a result of which, two children solutions are produced. To carry out this 
operation, the parents or mating pairs (each pair consists of two strings) are selected at 

: N Sy eee Wee 3 i 
random from the mating pool. Thus, у mating pairs are formed from а population of 
strings of size N. The parents are checked, whether they will participate 1n crossover 


by tossing a coin, whose probability of appearing head is pe. If the head appears, 
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r to produce two children. Otherwise, they will 
ant to mention that ре 15 generally kept 
ate in Crossover. 


the parent will participate in crossove 

remain intact in the population. It is import 

near to 1.0. so that almost all the parents can particip 

е а RU: .oim-flipping artificially: A 

The following procedure шау һе adopted LO implement COIT ae АТ = 2 T 

| ‘ | Үе ТЕ nber generator. [ 

number lying between 0.0 and 1.0 is generated using a random m i: E x e 

| | | · ГР Еі - equal to De, the outcome 'Oln- 

the random number is found to be smaller than 0) equal to pe, t 5 з head appears 

e : area it is false | head appears 

flipping is considered as true (that is, head), otherwise 1t 15 false, И the лева SEES 

the parents will participate in crossover. 

| ; | Жет Jdastad far crossover, the crossing site 18 

Once a particular mating pair has been selected lor crossover ce ЈЕ o Ү ai 

TEE | : qeu НИ ош - arer vine betweerl ATIC 
decided using a random number generator generating an integer yng He B 

От | ТЫ r of individuals participating 

L — 1, where L is the length of the string. The numbei of individuals pa I g 


in crossover can be probabilistically determined and 1t 1S found to bi equal LO D. 
Some of these 


There exists a large number of crossover schemes in the G A-literature. 
schemes are explained below. 

1. Single-point Crossover - We select a crossover site lying between 1 and 2—1, 
at random. where L indicates the string length. The left side of the crossover 
site is generally kept unaltered and swapping is done between the two sub-strings 
lying on right side of the crossover site. It is to be noted that children solutions 
will remain the same, whether the swapping is done on either left side or right 
side of the crossover site. However, we generally keep the left side unaltered and 
the bits lying OI the right side of Crossover site are interchanged. The Parents 
participating 1n a single-point crossover are shown below. 


Two children produced due to the single-point crossover are given below 


eevee Т (у 3 о 11 L. До = 05 O АЕТ cr 
КОЛЕ 0 0.101] 1 00. 1 1-0 150808 


2. Two-point Crossover - We selec y dp 
p C er - We select two different crossoyer sites lying between 1 


and L — 1, at random. The parent strings participating in this crossover are: 
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Two children strings produced due to the two- 
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point crossover аге: 
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v © site 9 are interchanged and the remaining bits are kept unaltered. The 


venerated children strings are oi 
спота childre 11 strings аге Piven below, 
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In this case, the bits lying between the sites 1 and 2: 3 and 4 are interchanged 


keeping other tact. hus, the children strings will look like the following: 
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Uniform Crossover - Uniform crossover [25| is a more general version of the 
multi-point crossover. Let us take an example of the parent strings shown below 


tO explain the principle of uniform crossover 
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In this scheme, at each bit position of the parent strings, we toss a coin (with a 
probability of 0.5 for appearing head) to determine whether there will be inter- 
changing of the bits. If the head appears, there will be a swapping of bits among 
the parent strings, otherwise they will remain unaltered. Let us assume that 
the 2-nd. 4-th, 5-th, 8-th, 9-th, 12-th, 15-th, 18-th and 20-th bit positions are 
selected for swapping. Thus, the obtained children solutions will look as follows: 


а 0499070 Ob зо 1908099 
б лал d O оде 1-10. 0 "ONT е 


Comparison of crossover operators [26, 27]: дойи 
The contribution of а crossover operator depends on its recombination poten- 
tial and exploratory power. Due to this recombination potential, crossover can 
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or multi-point a ind it using the crossover oper- 
int. Under these circumstances, mutation 
i н 23 . ш. 
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which changes 0 to 1 and vice-versa) can push a string from the local basin into the 
global basin (refer to Fig, 3.5) hy changing the-leftmost_bit from 0 into 1. Thus, 
mutation helps the GA LO search the global optimal solution. 

Го avoid a random search. mutation probability (py,) is generally kept to a low value, 
е range of Ur Lo | | 
implement А bit Wise mutation scheme. 







{ а i | | i 
It 15 Varied рх nerally in | | where L is the string length lo 


ks. 
à number lving between 0.0 and 1.0 18 created 
К d LALO number M neratot ut euch hil position || this number Cometes out to 
Ш. ss than CM ши Г? Phin ful i particular hit position then that hit will he mutated 
that 15 L Will be converted Into 0 


% 
ОГАП К OT 1 (+ А CHAT eT 


and ViCe- versn | || has heer observed that the 


in provi d USLE ar Чари mutation rate £0 


in = reproaucen Пп 5eC6IeCUs роба strings from the D pulation Lo form û mating pool and ITI 
| UR ен чер t pi perties between two parent strings and consequently, 

У | | l al changs AL A parti nnt bit position 
Once thi Ер | 1 БЕГИ у v i ed using thi BEL ГОГ namely repro 


Auction, crossover and ша! e generation of a GA is completed. Thi GA runs until 


Е. n b] а 2 : wa, - 
t П T я d 
' | i Р 
О [ kit E at | à d | 


generations or a desired accuracy 


The nerformance ol a | гү f thm depends signifi antlv on its Operators 
LIC POLNEM р | ch one out of these two operators 18 

FOSSOY e: TUN с-а t of study has been made to search the relative imi- 
more p х | а | | ——w tt han not Dan proved theoretically that 
portance of these tw a ! A d vice-versa. Spears [29 opined that а репе 
Em. | uc 3 "1 ront | ach as disruption and construction. It has 
per has to perform tw © поље и] than crossover in terms of its disruption 
been p ted out that mu! ' ] cH on in terms of the construction capability 
ipa tv. WIIerciu» ‹ Jd E fl 


m (СА) and Genetic Programming (GP) rely more Qu 


11 : |: den cx 
l b Greategies (ES) and Evolutionary Programming (EP), 


I L4 
UM Li Ver w fit [t L^ || r 
t T Г 
| я 211 A | CH || ne luba! Li 
d more welghtage 15 gl Vell ы * 


322 A Hand Calculation m Акра 
Tab! 1 calculation to solve an optimization problem given ^ 9 Ы АА. men 
lable 3.1 shows a hand саісшаиог = 5 at us try хапипе whether the з 
t е - soar T binary-coded GA. Let us try to exam 

је working principle Of à оле? 


;eneration to the 


next generation 
improve the solution from one & 


1 (3.5) 
Maximize у = У“ 


Subject to 


1< 2 


== 


1б. 


LA^ 
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КТ nciple of a binary-code 

| \ | alculation to explain the working princip 
Table 3.1: A hand calcula 












Expected count 
























| A. ‘(xr Pselectron 
| String | ` ]nitial - T Decoded | T b 
| No population | value | value z y: 
| | 10000] | 37 | 98 ~ 
| 2 011010 26 7.19 20 
| 8 010110 22 6.24 
4 111010 98 +) 14.81 m 
9 101100 44 11.48 ао 
29 001101 | 13 _ 4.09 | иы Од} a 1 | 
E т [sum 2, f = 17.91 
| du average f — 2.93 
| Im maximum f 3.85 | 








Parents 









Mutation 





| m MÀ a P = T ex 3 ee TRE T ч; "GT | 
Actual count. | Mating pool | Mating pair Crossover | Children 




























| Roulette wheel | e site | strings | 

M mio: | .3 | 100101 | 10/0101 101010 101010 
| 011010 6 | 111010 | 1111010 110101 110101 
0 111010 | | 011010 | 011/010 011100 011100 
2 111010 Б 101100 | 1011100 101010 111010 


| 111010 | 1181010 | 111100 





101100 4 | 111100 
101100 2 101100 | 1011100 | 101010 101010 





DW oT کڪ‎ EET ET 
Decoded T F(Z) 
value | value jede 

) 












42 11.00 3.32 ral 
53 13.62 3.69 
28 7.67 2 7 
58 14.81 
60 15.28 


42 





11.00 






sum У) f = 20.86 
average f — 3.48 
maximum f — 3.9] 
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The initial popuianonot binary-strings (of size 6) are ; 

‘fied using the proportionate selection (Roulette iy created at random, which are mod- 
(pe = 1.0) and а bit-wise mutation (eae | ıeel selection), a single-point crossover 
- orm 1 der A Pm = 0.03). The decoded values of the binary strings 
are det pane i5 depending on the range of the variable, its real values are calculated 
PE TORE binary-strings. Knowing the Values of the variables the function 
values have been determined. Аз it is a maximization Борас "M quee NE ae thie 
GA-strings аве nothing but the function values. Using e oration S ENS Неа of 
QUERN SES their probabilities of being selected in the ating Dee) are calculated. The 
mating pool has been generated then using the principle of Ropar оло selection scheme. 
The mating pairs are identified and all the pairs participate in the single-point crossover, as 
the probability of crossover p, has been assumed to be eduxit d. The children ви оне 
created due to this crossover are shown in Table 3.1 


As there are 6 x 6 = 36 bits in the 
Р F 1m f; 1 T) 
population and Pm 


0.03, there is a chance that only one bit will be mutated. Let us 
suppose that the second bit (from left) of fourth string of the population created due to 
crossover, has been mutated (that is, 0 is changed into 1). Table 3.1 shows that the popu- 
lation average has increased from 2.93 to 3.48 and the maximum fitness value has improved 
from 3.85 to 3.91, in one generation of the GA-run. Thus, it indicates that the GA can 
improve the solutions 


32.3 Fundamental Theorem of GA/Schema Theorem 


Ап attempt is made through this theorem to investigate the mathematical foundation of 
а СА. A schema (plural form is schemata) 1s a template, which may be present in the 
population of binarv strings created at random [8]. We try to predict the growth and decay 
Е « У У cet Lo L i Vd J | | : 
of a schema or schemata through the generations. 

Let us suppose that the population of binary-strings created at random, is represented 


as follows: 


і EON 040) 
i "o mU E f Tis f : ` ae ] 1larities 
If we look into this population of binary-strings carefully, we can fina some sip 


) at different bit-positions) among а number of strings 


in te ‘esence and ( та i 
(in terms of presence of la kemat (templates) are present in the above population 


us consider that the following two 5С 
of strings. 


ж 1l * 


Where x could be either 1 or 0. жо кс 
To proceed with the analysis of sc 
order О(Н) and defining length 


hema processing, We 
5(H) of a schema, W 


-— 


hich are defined below. 





“4 


Let us assume that this popu 
reproduction, crossover and mutation. 


e Order of a schema H, that is, О(Н ): 
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as the number of fixed positions 


It is defined 


E i — 2 and O(H3) = 9. 
(bits) present in a schema. For example, О(Н) = 2 and Os 


that is, б(Н): It 15 defined as the distance 


e Defining length of a schema Hs ‘tring. For example, (Ну) = 3-22] 
between the first and last fixed positio a string. . Xi FF 


and 6(H2) =6 —2 = 4. 


ns in 


lation of strings 15 modified using the operators, namely 


The effect of these operators on the growth and 


decay of a schema is explained as follows: 


1. Reproduction: Let us concentrate on a | 


yroportionate selection scheme, їп which 
jeing copied into the mating pool is proportional to 
its fitness value. Thus, the number of strings belonging to a particular schema Н at 
{ using the equation given below. 


the probability of a string for ! 


(t + 1) — th generation can be determine 


mi H.t--1) m H.t)N ce | (3.6) 


where m(H,t +1) and m(H,t) represent the number of strings belong to the schema 
Н at (t+ 1) — th and t — th generations, respectively; N indicates the population size, 
Ј(Н) represents the average fitness of the strings represented by the schema И at 
t — th generation, which may also be called as average schema fitness: V^ f indicates 


the total fitness of the strings present in the population 


The above equation can also be written in the following form: 


fi H 
ПАН. + 1) mí H.1)—— 94 
' | (3 [| 
FI "D р -— É Í " FE + Р " ® * 1 
where f = “ү 35 the average htness of the population 


Crossover: Let us consider a single 


ОТТ СТЕ ) ss i ror ryt the [5] 35 p ч i 
snes sche | i or the binary-strings. A schema 
survives, when the randomly selected crossover site falls outside th lef ing 1 | ћ 
alls outside е denning lengt 
" = 
qe 
ot destruction comes out to be 
ability and 
crossover survival probabil 


(5) of the schema. For this crossover, t] 


Scl le probability 
e ie ols) NO. | | 
equal to р. т, where p, and Г, are the crossover prol 
tively. Thus, the lower bound of 
like the following: 


string length, respec- 
iy р, can be calculated 
WES (Н) 
рз > 1 7 Pe — : 
N | | | L === | (3.8) 
Now, combining the effect of reproduction and cro 


equation as given below. 550Ver, we get the 


schema processing 


— Pc 


m(H,t+1)> тн, 0) 0). 
$ 


(Н) 
Lgs (3.9) 
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3. Mutation: Let us consider a bit. 
survival of a schema H, all the 
should not occur at the fixed bit 


wise mutation of the binary strings. To ensure the 
fixed bits must survive. In other words, mutation 
5 to protect the schema. 

Let us suppose that the probability 
by pm. Thus, (1 — pm) | 
the probability of survi 
be given by the follow 


of mutation/destruction for each bit is denoted 
represents the probability of survival for each bit. Similarly, 
| val considering all the fixed positions/bits in the schema will 
Ing expression: 


Ps = (4 = Ре — pe), OH) 
A (1 — Pm jeu) 

As Dm « 1, p, can approximately be set equal to (1 — O(H)p,). 

Considering the contributions of all three operators, namely reproduction, crossover 


and mutation, the schema processing formula can be written as follows: 


m(H.t+1)> m H 82211 p, rr — O(H)pym). (3.10) 
F AT рт) 


From the above equation, it is interesting to observe that the schemata having low order, 


short defining lengt h and whose ntness y alu 5 are more 1 han the average ht ness ot the popu- 
lation, will receive more and more copies in future generations. They are called the building 


blocks of GA-search. This theorem is also known as the Building-Block Hypothesis of 
a GA. 


3.2.4 Limitations of a Binary-Coded GA 


Binary-coded GA is the oldest of all versions of the GA. However, it has the following 


limitations: 


1. If we need more precision in the values ol the val lables. we will have to assign more 
number of bits to represent them. To ensure a proper search in a such situation, 
population size is to be kept to a high value and as a result n which, computational 
complexity of the GA will increase. Thus, а binary-coded GA may not be able to 
yield any arbitrary precision in the solution. This problem can be overcome using a 
realcoded GA. the principle of which has been explained in the next chapter. 


9. In binary representation, if we want to move from one number to the next or previous 
number, the number of changes to be made in the bit-position may not be the same. 
For example, the numbers 14, 15 and 16 are represented by 01110, 01111 and 10000, 

| d thus, five bits are to be changed to shift from 15 to 16, whereas it 

e of only one-bit to move from 15 to 14. This problem in binary- 

iff problem, which may create an artificial hindrance 


respectively an 
requires a chang 
coding is known as hammi 


to the gradual search of a GA. This problem can be eliminated using & gray-coding 


scheme, in place of a binary-coding scheme. 
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possible by making only one 


T f numbers 18 
shifting of numb well a5 gray-codings of 


In a gray-coding system, the „> уне wet ina binary da 
ance in the bit-position. Table 9.4 5557 
change in the bit-pos 


the numbers starting from 0 to 15. 


r Е, 
s ste г from U to 15 
rav-codings of the numbers starting 
Table 3,2: Binary and gray-codallHip 


-— 


"Number | Binary-code | Gray-coce 
or 0000 0000 
| 0001 00 J 
2 0010 001 і 
3 0011 0010 | 
4 0100 0110 
| 5 | 0 0111 
| 6 0110 0101 
| [ | 0111 | 0100 
8 | 1000 | 100 
9 | 1001 | 1101 
10 | 1010 111] 
| 11 1011 1110 
12 1100 1010 
| 13 | 1101 1011 
| 14 | 1110 | 1001 
19 | 1111 1000 
ڪڪ‎ SS a 


3.3 GA-parameters Setting 


The performance of a genetic-search depends on the amount of exploration (population 
diversity) and exploitation (selection pressure). To have an effective search. there 
must be a proper balance between them and to ensure this. the GA-parameters, such as 
population size, crossover probability p. and mutation probability р, are to be selected 
in the optimal sense. The interactions among themselves are to be studied to select their 
optimal values. It is obvious that the optimal GA-parameters are problem-dependent. Sev- 
eral trials have been made to understand the mechanics of interactions of GA-parameters 
using empirical studies, Markov chain analysis, statistical analysis based on Design of Ex 
periments (DOE), and others. In this section, a met hod of determining the near-optimal 
GA-parameters has been discussed. | 


Let us suppose that we have decided to vary the GA- 
bility pe, mutation probability Pm and populati 


parameters, such as crossover proba 


| | on size N in the ranges of (0.6 to 1.0), (0.001 
to 0.011) and (50 to 150), respectively, while à particular minimization Hw 

be varied in the range of (50 to 
scussed below (refer to Fig. 3.6). 


solving 
5 быш will 
four stages as di 


Moreover, the maximum number of generation 
150), say. This experiment is conducted in 
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Figure 2.6: Results of the parametric study conducted for determining the optimal GA-parameters. 


e Stage 1: We generally vary p. at first. within its range, after keeping other parameters 
fixed at their respective mid-values. Thus, р, is varied irom 0.6 to 1.0 in steps of 0.1 


say, and Dm, N and Gmar are kept fixed to 0.000, 100 and 100, respectively. Let us 


suppose that the fitness value is found to be the minimum for a particular value of 


De, say 0.9. 


e Stage 2: We fix the values of pe, N and С to 0.9, 100 and 100, respectively and vary 
Pm starting from 0.001 to 0.011 in steps of 0.001. Let us assume that the fitness value 
is seen to be the minimum for рт 0.002. 


e Stage 3: The values of Pc, Pm and Стах are kept fixed to 0.9, 0.002 and 100, respec- 
tively and the population size is varied from 50 to 150 in steps of 10. Let us suppose 
that the fitness is found to be the minimum for a population size of 110. 


* Stage 4: In this stage, Pe, Рт and N are set equal to 0.9, 0.002 and 110, respectively 
and experiments are carried out after setting the number of maximum generations аї 
different values (starting from 50 up to 150 in steps of 10). Let us assume that the 
minimum fitness is obtained, when the GA is run for a maximum of 120 generations. 


Thus, the optimal GA-parameters are found to be as follows: p. = 0.9, р = 0.002, N = 110 
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22 1с ee eee eee 
thod may not be able to determine the sef 
ay of finding the suitable GA-parameterg 


It is important to mention that the above me 


of true optimal GA-parameters. It is an easier W ЖА. MS ies mee E 
approximately. Let us assume that the range of each GA-parameter 18 divided ШОШ 


equal divisions. Thus, only 10 different values аге considered for each GERA BIDET while 
carrying out the parametric study. It is important to note that in the AROE methog 
experiments are carried out only with 40 (that is, 10 + 10 + 10 т 10 = 40) sets of GA. 
parameters, out of a maximum of 10* possible sets. Thus, there is no guarantee that the 
above method will be able to yield the globally-optimal GA-parameters. 


3.4 Constraints Handling in GA (^ 


Several attempts have been made by various investigators to solve the constrained optimiza- 
tion problems using a СА, in which the fitness function of GA-solution has been expressed 
іп a number of ways to tackle the constraints after ensuring its proper search. It is to be 
noted that the constraints may be of different natures, such as linear, non-linear, equality, 
inequality, and others, which basically divide the search space into feasible and in-feasible 
regions. 

A constrained optimization (either maximization or minimization) problem may be ех: 
pressed as follows: * | 


( )ptimize |Х (3.11) 


subject to 


eX) 04 —1,2,....m (3.12) 
AIX) Оз —1,2 71.9 (3.13) 
where n and p represent the number of Inequality and equality constraints, respectively 
= (11,22, ..., Tm) are the design variables having the bounds given below br | 
11 | | M ] M 
zn Ta zn LI 
JUI o. тат 
q m = Чт ~ 1 pi 


It is important to mention that the above constraints could be either linear er ~ | 
in nature. Let us use the notation ape Pe m5... ‚@, to SSE кз SE поп-Ипеш 
as non-linear constraints together. Thus, q becomes equal to (е p а 

A number of constraint handling techniques have been No Kae P 
tion approach, method of maintaining a feasible oh ID 
tions, approach aiming at preserving feasibjlit | M 
the objectives and constraints (30, 31, 32], and others Outc 
penalty function approach is the most popular опе, d 7 о 


l, such as penalty func- 
| on over infeasible solu- 
y of solutions, approach separating | 
f all these techniques, the 
аз been discussed below. A 





| 
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detailed discussion on all the above 


| | Approaches is he | ! 
readers may refer to [30, 31 PI es is beyond the scope of this book. Interested 


00 m. 
‚ 32] for the above purpose. 


3.4.1 Penalty Function Approach 


In this approach, the fitness function of 


"Wa: | a solution (say i-th) is expressed | lt Fa ЇЙ 
objective function as follows: (вау ) is expressed by modifying its 


FX) = fi(X) + B, (3.14) 
rm c а | 
where P, indicates the penalty used META vU c г ST 

penalty used to penalize an infeasible solution. For a feasible solution, 


P. jis set equal to 0.0, whereas for an infaacik 3 3 , 
1 l eas tor an infeasible solution P, is expressed like the following: 


q 
P= CD (Фоо), (3.15) 
К=1 
goare C indicates the user-defined penalty coefficient and {dix (X)}* represents the penalty 
term for k-th constraint, corresponding to i-th objective function. It is important to mention 
that the above penalty could be either static or dynamic or adaptive in nature, which are 
explained below. 


Static Penalty [33]: 


In this approach, amount of constraint violation is divided into several levels and penalty 
coefficient Су (for r-th level of violation of k-th constraint, where r = 1,2,...,U; U indicates 
the user-defined level of violation) is pre-defined for each level. Thus, the fitness function 
for i-th solution can be represented like the following: 


| а | 
| Е \ x I fi | X: ) + у) C т { Oik ( х ) ) (3.16) 
к=] 


The main drawback of this approach lies in the fact that its performance is dependent 
on a number of user-defined parameters, which may be difficult to determine beforehand. 


Dynamic Penalty (34]: 
In this method, amount of penalty varies with the number of generations. The fitness of a 
GA-solution is calculated as given below. 


| F(X) = A) + (C x 2* 97 lex OO! (3.17) 
| k=l 


where C, a, B are the use 
Penalty term is dynamic in na 
| ie algorithm with dynamic pen 
es. However, its performance 


r-defined constants, ¢ represents the number of generations. The 
nature, as it increases with the number of generations. — - 
alties is found to perform better than that using static 
depends on the pre-defined values of C, а and 8, the 
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Adaptive Penalty [35, 36]: 


ЕТ ;ed-back during the search. 
' h. the penalty terms аге updated by taking feed-back » Ne 
i this approach, the репа у >= ad in a number of ways by various Te 
= hee х * jf idaptive penalty has been developed in & E ls | A in which 
je. concept ol adapwYw pee f — 1 far 261 їс one of such trials, in. Waic 
hers 32 Bean and Hadj-Alouane s approach (35, 36] is one of suc 
searchers |04]. eal ais AE 
the fitness of a solution 1s expressed as follows: 


Яс... 2 (3.18) 
F(X) = ЛО +A) 2 (04007; M 


к=] 


-ameter A(t) is updated like the following: 


where t indicates the number of generation. The рат 
(jx Alt), ТОГ (Clase | 
I] 4 > ( 7) 10 
VER 4-1 i ب‎ Nie) К (naa 9 oP 2) 
AVE + | јо X ЛЕ], | | ы 


where Di F 02 апа Gi, Oo > 1, Case 1 represents the situatio) where the prev 10081 found 


= А 5 j | |" Ó 5 ] i- ЕТТ. T IT 1T M Tal је 
Dest individuals always remain feasible and Gase 2 Indicat hose situations, in which the 

: ee | | ' oett дели that 
obtained best individuals are always seen to be infeasible. It is Important to mention tiat 
105 performance depends on the selection 01 0 ПА 172 les, чп May Dt difi ult to 


pre-determine 


3.5 Advantages and Disadvantages of Genetic Algorithm 


Genetic Algorithm has a number of advantages over the traditional optimization tools but 


it has some disadvantages too, which are discussed bel 
Advantages of GA: 
It is important to mention that a GA can over« ome almost all thi limitati 


tations of traditional 
optimization tools. It has the following AU Vvaritages 


| A GA starts with a population of initial solutions created at random Out of all these 
fav МИЛО. ut OI all these 


tt | 
IO D: 


initial solutions, if at least one solution falls in th: 


bd. li 


asin, there is a good chance 


DM | iInportant to mention that initial 
зоршацоп may not contain any solution Јуте in the sl ‚ 1 | 
PO} У mung MULIOn tying In the global basin. 


for the GA to reach the global optimal! solution. [t is 


In that case, if 
| MM. m T | that case, 1 
the GA-operators (namely crossover, mutation) can push at least one s luti int 

Rp ува Vea SEEN RR Bou NM | EST. vri oot а WiC SOLUTION into 
the global basin, there is a chance that the GA will find the global ontim: | 
Thus, the chance of its solutions for heme + | &Obal optimal solution. 

s, пе chance Of Its solutions for being trapped into the local minima is | 
u une local m là is less. 


2. They can handle the integer programming or mixed-integer programmi 
т. ~ У МИНА TORT: ning prohle 
effectively. 5 ng problems 


3. AS gradient information of the objective function js not required by a GA. i 
optimize discontinuous objective functions PE , It c 


АП 
also. 


4. It is suitable for parallel implementations. 
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rhe same GA with a little bi ~ ail 
hiena re little bit of modification in the string can solve a variety of 
problems. thus, it is a versatile optimiz 


ation tool. 


Disadvantages of GA: 
Alth 


3.0 


j 


= 


6. 


‚ А binary-coded СА suffers from the 


wh the GA has а numbe adv: ' | 
| iber of advantages, it has the following disadvantages: 


computationally expensive an 
П] y expensive and consequently, has а slow convergence rate. 


[4 "rud ел. m! ‚лл РЦ РЕЦИ i + А 
[t works like а black-box optimization tool. 


| - * 18 ПО Hik {| 2 Te Cum X те ae ч а i 

here is no mathematical convergence proof of the GA, till today. 

il ЄТ rri GT | Р M ” # Г r п = Les == : А = | 
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) TT prt сл 
A. kh d LI 


Summary 
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The working cycle of a GA has been explained. In a СА, the solutions are modi- 


ned using different operat such as reproducti jn. crossover, mutation and others. 
Reproduction select d strings from a population and copies them in the mating 
pool. In crossover, tnere 15 ац exchange of propert ies between the parents and às a 
E й + || ë у | | 


result of which, the children solutions are created. Mutation brings à local change ın 
the current solution and thus helps the sı lution to come out of the local minimum, 


if anv. Thus, the chance ot its :olutions for getting stuck at the local minima is less. 


l'o ensure an effective search of the GA, there must be a proper balance between 


its population diversit 7 exploration) апа selection pressure (exploitation). An ар- 


such as crossover probability, mutation probability, 


population size, maximum number ol generat 1005, is to be determined through a Care- 


propriate set ol the parameters, 


ful study for the above purpose. 


Schema theorem of the binarv-coded СА 1s able to explain the reason behind the fact 
JIE Ё j LUI G i DE E S ё Г 


the solutions from one generation to the next. 


that it can improve 
А binary-coded GA cannot obtain any arbitrary precision required. It is 80, because 
s baro: 1 B ban of bits are to be assigned to represent à variable for obtaining the 
aree nul › | DI GT =. 07 = . : TUA ` i | 
h à ier . the number of bits in the GA-string increases, 108 computational 
jetter precision. As the n | · which. the GA will become slower 
complexity will be more and as а result of which, the V. 


Hamming Cliff problem, which can be eliminated 
using a gray-coded GA. 


|| drawbacks of the traditional methods of optimization 


ally expensive. 


А GA can overcome almost al 
but it is found to be computation 
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enalty function approach of constraint handling used 


be either static or dynamic or adaptive jp 


An introduction is given to p 
along with the GA. The penalty term тау 


nature. 
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ely repri duction, crossover and 


Explain briefly the role о! 


mutation used in a GA 


yetween selectlon pressure and 


J <= 
TA ay ley Ё Y epe уте 4 T | + ' 
рор па Ё VoD У | і | Lit |! 
= 
Arg шпногтп i - i^i th D | | LT] probabili V 
» 7 = 1 
OLU- J спе запи EXA || 
G'ar i1 "А 
Explain etisi 5 еп | | ( | 


LCLILIVIITI 


WI! - "x | + | 
ght ата TU = іса 1 r3 Тл» aa || Y 1 
i4 y ЈА» тет Да t ДЕЈ YELL VUL XI ы | | ali 
Г ] OI mutation 

v M i 
ргорар Li ] АА LII] Ld 

"r 11 mont iT" i „ака xf Aij | А + 
Do J UL + Н АЕ cd 1 л thy =. oded ( тга UU d BIT | 


dE iles - Lt E rm Ж. P у ГУ] Т а 
Explain the Building-Block Hypothesis о! 
(Па TA ог ат (LÀ | 

al you declare (зА a пора! Ooptimizer *? 
State the advantages and disadvant 


Use a binary-coded GA to mi 


г 12—212 — 12 4 тата 


a rangdor Population ol size N 


пиле the function | 


in the range of 0.0 € i,r < 5.0. Uso 


point crossover with probability p 6, a single 


| Џ anc пе || С! : 
each variable and thus, the GA-strino will 1 "Wtauon. Assume 3 bits for 


be 6-bits long. Show | 
a hand calculation. ong. SNOW only one iteration by 


An initial population of size N = 10 of a h; 
“ey мани dv 40 е) а binarv cod 2 
A y-coded GA 


IS created at random as 
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expected number ol strings to be represent: 


the end of first generatii 


MI, 4 


and ch bil -WISE mutation ol probability D. 


13. А closed-coil helical spring (refer to Fi 
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produced in it di 


le to twisting апа 1i 


The volume of the spring can be determi 
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the coil diameter, Ne denotes 


1 "A ‘ac (TI ts 
ы the ter of the wire, D represe | | 
ates the diameter of t he spring can be calculated 


where d indic | 
The developed shear stress in t 


the number of active turns. 
using the expression given below. 


gC, FD 
Tdeveloped “ТАЗ 


4c 4 9.913 P. F represents the maximum working load. The 
where Су = 46-4 + у 


ranges of different Variables are kept as follows: 


and C = 


0.5 < d < 0.8, 
15« D < 8.0, 


Assume that the density of spring materi: М is represente d by p 


a =ч ] | fe ya T ] Р рът асетттит а i 1 и ғ n ry | 
• Use a binary-coded GA to solve the constrained (after assuming a suitable value 
of S) and corresponding un-con strained opi mization DI blems. The resi val 
ables are assumed to have the accuracy level of 0.001 (approx) 
Develop a compute: program ‹ f the binary-coded GA and solve the above prob: 


lem using different forms of reproduction scheme (such as р! rtionate selection, 


Te Ди] va (ooa, is & | * н 
ranking selection, tournament selection), crossover operator (namely single-point 
crossover, two-point crossover, uniform crossover) and a bit-wise mutation. 

a Г ^ Y "o 
Hints: To determine the leneth of GA-substrine 


required for representing 8 
variable т after ensuring an accuracy level of « , follow 


| LX. | | [ € "c S MI gl Ver below. 








Chapter 4 


Some Specialized Genetic 
Algorithms 


To ensure good precision in the values of real variables and overcome Hammung Cliff prob- 


lem, а binary-coded GA may be replaced by a real-coded GA, the principle of which has 


been explained in this chapter. A GA is found to be computationally expensive and it may 
not be suitable for on-lini impiementations. Several attempts have been made to develop 
some faster (GAs, namely micro-GA Visualized Interactive GA (VIGA) which are 
discussed in the present chapter. Scheduling is a special type ol optimization problem, in 
which not only the positions of the elements but also their order and adjacency are im- 


portant. Thus. the conventional crossover operators discussed in the previous chapter may 
i | | : А 
not be able to tackle this problem efficiently. Scheduling GA requires а special type 
of crossover operator. Lhe principles of some of these Spe jalized crossover operators are 


explained in the present 


41 Real-Coded GA 


A real-coded GA can overcome the dit ulties faced by a binary-coded GA (refer to Sec- 
tion 3.2.4) to carry out optimization ш da continuous зеагсп Space. For the real-coded 
GA. several versions of crossover and mutation Operators have been proposed by various 


researchers. The principles of som of these operators are explained below. 


4.1.1 Crossover Operators 


A large number of crossover operators, namely Linear Crossover [37], Blend Crossover [15], 
Simulated Binary Crossover [38], and others, have been developed for the real-coded 
СА. 








SOFT COMPUTI, 


SS a ы 7 E 


Linear Crossover: M | 
To explain its principle, let us consider the 
crossover. They produce three solution 
_0.5Pr; + 1.5Pra). Out of these thre 


Жы - Oo] [t FF 
[t was proposed by Wright in 199] 187) Hos 
two parents - Pr; and Pr аге participating a 
Ре 1 т Тг : 
as follows: 0.5(Pr; + Pro), (1.5Pri 0.5Pr»), 


‚ the children solutions. 
solutions, the best two are selected as the children 


Example: In, — 15.65 Рт» = 18.83. 
Let us assume that the parents are: Pr; = 19. 9, А endo be like he following: 
Using the linear crossover operator, three solutions are IOUI F 


0.5(15.65 + 18.83) = 17.24, 
1.5 x 15.65 — 0.5 x 18.88 = 14.006, 
—0.5 x 15.65 + 1.5 x 18.83 = 20.42. 


The parents and obtained solutions are shown in big. 4.1. 


Solutions 
1406 1724 20.42 


Figure 4.1: Th parents and their solutions obtained using linear crossover. 


Blend Crossover (BLX - a): 


This operator was developed by Eshelman and »chaffer in 1993 [15] 
parents — Pr; and Pr», such that Pri < Pro. It creates the Ee 


hz =f > ЕЈ ми ЈЕ Jo VL [ f a 
the range of (Рт; — a( Pra — P уј 1 Рт: + al Pra Ртү)}|, wl 


jt T 
o not come out of tl 


Let us consider tw 

children solutions lying E 

| | iere the constant a is to b 

selected, so that the children solutions d le range. Anoth ter^ 
~ e EEE Ј er paramet 

i EH defined by utilizing the said а and a random number r lying in the range of (00 

1.0) like the following: RITU ie Pa | 


m, 


-(-4-20)r а. 


The children solutions (Ch,, С ha) are determined trom the т 


Ch; — (1— Рт; + + Рт. 
Ch, = = 7) Рт + YETI., 


Jarents as follows: 


Example: 
Let us assume that the parents are: Pr, 
Assume: a = 0.5, r = 0.6. 


| | 
The parameter y is calculated like the following: ] 


= 15.65, Pro = 18.83. 








Wm +. “Jin, : 
(1+2@)гк -a= (1 +2 x 0.5)0.6 — 0.5 = 0.7 


Tha children solutions are ae | 
[he chudren soiitons are then determined as follows: 


Chi = (1 — 7)Pri + Pry = 17.876 
Che = ite У) Pre | ҮР; 


1 


16.604 


|| 
- . s апа obtained childran arn : 
rent: tained children are shown in Fig. 4.2 
‘=| Sn иге 
Children 
16 m |7 576 
| 
| AA | | 
iU Ü | | 34 () 
1% 65 |x &3 
rel 


Simulated Binary Crossover (SBX) (38, 31 


каз proposed by Deb and Agra in 1995 |38|. Its search power 15 represi nted with the 
help ot а probability disi | generated children solutions from the given parents. А 
spread factor o has been introduced to represent the spread of the children solutions with 


[um i-r 
ы ы 


(4.1) 


vhere Pr. Pr represent the parent p nts ind Chi Che are the children solutions Three 
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• Case 1: Contracting crossover (à the spread of the children solutions is less 


Tq FA ASt or ther reTits 
LI ПА UI Lit pd LA VA 


ы Сазе 2 Ex ја ding crossover а > |] the spread of the children solutions 15 more 
: : храги Ing CrOssty. 


than that of the parents 


AR ` var [n = 1\-the spread of the children solutions is ехас 
* Case 3: Stationary crossover (a = 1) the spread o n solutio у 


the same with that of the parents. 

creating children solutions from 
refer to Fig. 4.3): It is important 
depend on the exponent q, which is a non-negative 


In Deb and Agrawal [38], the probability distributions for 
the parents have been assumed to be polynomial in nature ( 
tO Mention that probability distributions 
= ae А 
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from the parents vs spread factor 


Figure 1.3: Probability distributions for creating the children soiutions HOM 


а |38| 


real number. For the contracting crossover, the probability distribution is given by the * 
following expression 4 | 


| Áo | (VF \ | | Ai 
( (У | ig = | ја“. (4.2) 
| | 


On the other hand, for the expanding crossover, it is | xpressed as follows: 


Es : | | | 
Ez(a) = 0.5(9 +1) ==. | (4.3) 


{Fi 


It is also interesting [ Uu note that [01 small vales of Ч, the children are renerally far away 
[ t! ], they are created in the close neighborhood 
Oo t је аге its. “т T ` yu s à) Ure TI P^ ct y p T „м e Га mS can в ж Е = 

pe П | ЈЕ 4.3 sho х) Ше Variations ol probability distributions [ОГ different values 


of g (say 0, 2 and 10). It is also important to state that + | 
| 1 ЈЕ MED, в also Important to state that the area under the probability 
distribution curve in the contracting crossover zone (that is. [T (а 
2 du па 15, Jag C(a)da) and that in the 
“(ас јаа) are found ео 
a=] jda) i ound to be equal to 0.5 eac 
Е Маре л | : .9 each. 
The following steps are used to create two children so] 


lutions Ch, ; у 
parents Pr; and Pro: ~hy and Cha from the 


| from the parents, whereas for high values оѓ, 


expanding crossover zone (that is, | 


e Step 1: Create a random number г lying between 0.0 and 1 0 


e Step 2: Determine а for which the cumulative probability 


f 


0 С(а)аа =r, if r < 0.5. 
and 


, 


E Jı Ex(a)da = 


Даш 0.5, if г> 0.5. 








| өтер S Knowing the value of a, the children solutions are determined like the 
| с P ke: th 
following: 


Ch; = 0.5[(Pr1 + Pro) – а |Р — Pn] 
Cho = 0.5[(Рт + Pr2) + а [Ern — Pri]. 


xample: | ы DRE. Dre 1R aA тое HS PEN 
i as assume that-the parents are: Рту = 15.65, Pro = 18.83. Determine children sO 
e 3 бог 


tions using the SBX. Assume exponent q — 9 


Let us consider that the generated random number т is equal to 0.6. Азт 0.5 
eX ص‎ 
calculate а, such that 


n Er (ada = T — (0.0, 


1.е.. bap 0.59 111—7 аа == LRL 


After solving the above equation, we get о 1.0772. Thus, the children solutions are тшпа 
to be as follows: 


Y VE Г) ГУ - den m _ РЕКЕТ 
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Ch јр = | | ad | Í d | | 1 [ T ) | = (Y | [ "7 2 і | ] || = | rn 4 Gog [ 


The parents and their children obtained using the simulated binary crossover are shown m 


Fig. 44. 


25.0 





| 10.0 | 


i 15.65 18.83 


Parents 


Figure 4.4: The parents and their children obtained using simulated binary crossover. 


4.1.2 Mutation Operators 


ever | У am A by various 
Severa] Versions of mutation operator have been proposed for the real-coded GA b) 
Searchers. Some of these are explained below. 
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lying be 


where r is а random типос! 
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tion defined by t^t iser 
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Example р; | 
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Deb and Goya 0! proposed а muta lon opera г Da 
following steps are co | | the mutated 
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| solution using th 


| (т — 0.5)A; 


etween 0.0 and 1.0, A is the 


35i d ОП polynomial 
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e rule given below. 
(4.4) 


value of pertur- 


maximum 


? | е mutated solu- 
= 6. Determine the mutat od solu 


| distribution. The 
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| - Р a ~ г | cml FT ш 
lution from an origita solution: 
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where Ómar is the user-defined maximum perturbation al 


5 [| | Реј а Вал и РТУ y + 
and mutated solutions. 


Example: 


Let из assume the original parent solution Prorioinal = 
Fi LT ü 


7 "ma ing r T A | 
PF dated, considering т = 0.7, q = 2 and ómar = 1.2. 
Corresponding to r = 0.7 and q = 2, t 


iy | 


| 
T 


1 d 


lowed between the original 


19.0. Determine the mutated solution 


he perturbation factor 6 is f 
d ation factor ó is found to be as follows: 


б=1— [2(1 - г)] 9+1 = 0.1565 


The mutated solution is then determined from the orig 


inal solution like the following. 


є‏ = | ر 
Pr molated = Proricinal t 0 X + = 15.7878‏ 
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4.2 Micro-GA 


As the string length of a binary-coded Simple GA (SGA 
in the values of the variables, it is advised to run the 
The larger population ensures a better schema pro 
chance of occurrence of premature convergence. 
be obtained but at the cost of a slow convergenc 
line optimization of most of the real 


) increases to ensure better precision 
GA with a large population of solutions. 
cessing and consequently, there is a less 
Thus, the global optimal solutions may 
e rate. The SGA cannot be used for on- 
| - SE -world problems (such as on-line control of а mobile 
robot), in which the objective function may change at a rate faster than the SGA can reach 
the optimal solution. Realizing the need of a faster GA for on-line implementations, а 
small population-GA (known as micro-GA ) was introduced by Krishnakumar [16], which 


is basically à binary-coded GA. The working principle of the micro-GA is explained below 
in steps. 


e Step 1: Select an initial population of binary strings of size 5, at random. 


e Step 2: Evaluate the fitness of the strings and identify the best one. Mark the best 
string as string 5 and copy it directly into the mating pool. It is known as elitist 


LI س‎ 
strategy. Thus, there is a guarantee that the already found good schema will not be 
lost. 


Step 3: Select the remaining four strings (the best/elite string also participates in 
reproduction) based on a deterministic tournament selection strategy. 


e Step 4: Carry out crossover with a probability pe = 1.0 to ensure the better schema 
processing. The mutation probability is kept equal to zero. 


e Step 5: Check for convergence. If the convergence criterion is reached, terminate the 
program. Otherwise, go to the next step. 


e Step 6: Create a new population of strings of size 5 by copying the best (elite) string 
of i. ene ted population and then generating the remaining four strings at 


random. Go to Step 2. 


ian t] entional SGA. Initially, all five strings are 
i l to be faster than the conventio: a | 
| E. x TA and new four strings are added to the population in all subsequent 
ан аф Th fore, the necessary diversity has been maintained in the population 
[^o Ири Le осы the chance of pre-mature convergence of the algorithm cannot 
uring its search. Nowevet, | 


be totally ignored. 


ef alle 
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"interactive GA (VIGA) (41, 42] has been developed to serve the fo 
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A | DV Ia hiective function to be opti. 
e To investigate topological information of the surface of objective tun pti 


mized. 


e To accelerate the GA-search. 
d to map the higher dimensional 


La VIC | anping methods are use 
develop the VIGA, some mapping methods a | RESI 
| i Some of these mapping tools are 


data to a lower dimensional space/plane tor visualization. 


4 


discussed below. 


4.3.1 Mapping Methods 


Our visualization capability is restricted to three dimensions (3 — D) only. We are unable 
to visualize the higher dimensional (more than 3— D) data. Thus, these higher dimensional 
say puni data are to be mapped to 2— or 3 — D for visualization. A number of 
methods had been tried in the past to map the data from ch higher dimensional Space toa 
lower dimension. These mapping methods can be ‹ lassified into two groups, namely linear 
and non-linear methods. The linear methods include principal component analysis, 
least square mapping, projection pursuit mapping and others [43, 44]. On the other 
hand. there are some non-linear mapping methods, namely Sammon's Non-Linear Map- 
ping (NLM) [45], VISOR algorithm 461. Self-Organizing Maps (SOM) [47], and 


others. Out of all these mapping tools, only two ari discussed below, 1n detail. Moreover, 


the principle of another mapping tool, namely SOM will be discussed in Chapter 8. 


A. Sammon's Nonlinear Mapping: 

Sammon's Non-Linear Mapping (NLM) is a distance preserving technique, in which the 
error in mapping 15 minimized through 4 number of iterations using a gradient descent 
method [45]. 

Let us consider N vectors in a multivariate data space of L-dimension, which are denoted 
by X;, where 1 = 1, 2,---,N. Our objective is to map these N-vectors from L-dimensional 
space to 2 or 3-dimensional space. Let us also suppose that the mapped data in 2 or 3 
dimension are represented by Yî, where i = 1,2,...,N. The scheme is shown in Fig. 4.5. 


mapping 


L-dimensional space 


2 or 3-р space 


N- vectors | 
= 


ХЕМ | 





N-vectors 






Figure 4.5: Mapping from L-dimensional space to 2 or 3 — D space using NLM 
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| 135 
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The mapping technique can be described as follows: 
i r AI лије о in * ет" в 
1. Create /V vectors in 2-dimensional plane at random 
2 d*. be the Euclidean distance hetween twe > E. ; | 
fi: Let ij the Euclide an distance between two vectors X; and X; ina higher dimen- 
sional space and d;; is the distance between the corresponding two mapped points Y; 


r1 4 II ra 3 таг {сал 0 = 7 vv 0 ~ LU Р : : m А Fg. 
and Ys, in a lowe (Say 2) dimension. For an exact mapping, the following condition 
is to be satisfied: 


di; = dis. (4.6) 
3. Let us assume that E(m) represents the mapping error in m-th iteration, which can 


be expressed mathemat ically as follows: 


AJ N | i (39 
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Ex iy E [^17 пи NS Р. 
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7 T, ATP = f N19 
where C = > =. SS. lie а. and di: ;(m) = V k= Yak) yjk(m)] 
22i = =) = = 1) e j 
4. The mapping error 15 minimized using a steepest descent method, for which the rela- 
tionship between the m-th and (m4-1)-th iterations can be expressed like the following: 
Урд (m r |) = Yog(™) d (МЕДА рат), (4.8) 
| — ina im the range of 0.0 to 1.0, and 
; 2 fateor which varies in the range of 0. | 
| where МЕ is а magic factor, whic 
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• Determine the pivot VeCLOI Vi such that distance 


d( Vi, M) mar’ (ап, M )) 


E 


e Determine the Pivot vector У; such that di tani 
- NIIT, = о LUS Lel up 
d( V2, Vi) = тат dlu, V;)), 


e Determine the pivot vector Va enc | 
pivot vector V4 such that distance 


d(V3, V2) = mazi 2(4(0; V;)) and div. Ls 
<> 2)) ап d(V5, Vi) mazi  (d(v;, V, )). 


i OTIC] = | | я 
Бар L-dimensional space, 
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2. The pivot-vectors — Ext 


Р, Р. r Ti T , i в - с 
2, £3 are located in 2-dimensional plane corresponding t 
the vectors Vi, V, | па! pie терс g to 


i ү. = = 1 F Va pea T Lj А Н a 
‚ V3, respectively, using the information of Euclidean distance., 


3, Mapping of the remaining (N — 3) points will follow this approach: 

e Supposing that a particular point V of L-dimensional space is to be mapped 
to a corresponding point in 2-dimensional plane. The lines Vi V? and ЊУ are 
consideren. Two perpendicular lines are drawn from the point V to the straight 
lines Vi апа Њу. Thus, the points Кү and К» are obtained on the lines Vj V2 
апа WV, respectively. ; 


n2-— the poi Sills a | | че. 
I D, the point D, is located on the line Pj Ps, such that D; divides the line 
P, P, in the same proportion as K, has divided the line 1 И (in L-dimensional 
space). Similarly, the point Də» is determined on the line P; Ps. 

e [he perpendiculars are drawn at the points Dj and D» to the lines РР» and 
3 л сту oe ears = FT A. - " rix. 2 - 3. - _ - =. 

5 Рз, respectively. They intersect at the point P. Thus, the point У of L- 


dimension is mapped to a point P into 2 — D. 


It is interesting to note that in this algorithm, the higher dimensional data are mapped 
into a lower dimension in one iteration only. It is : xpected to be a faster algorithm, as it 1s 
a non-iterative one. 


4.3.2 Simulation Results |49 


Figure 4.7 shows the surface plot of Schaffer’s F1. 








| | С 0 5 
0.495 f i i | 

0.49 | у ЊУ. 
0.485 | x ч | 

0 А 4 8 Г ff j 22 © | T. T 

M 7 100 
А fs 
7 У 50 
210 а= Р К, 0 
-50 cro У 
=. < -50 
50 ^ 


(a) 


Figure 4.7: Surface plot of Schaffer's F1 function. 
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It is mathematically expressed as follows: 


sin” 512? Ed 0.5 (4.9) 
= ПД = 
J ZU 104 0.001 & 22)? 
of this function are to be mappa 


CIE Be ee he surface 
Let us suppose that 200 random points lying on the 2—D using Sammon’s NLM 


to 2— for visualization. Figure 4.8 shows the mapped data in 
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Figure 4.8: Comparison of the mapped data obtained using (a) Sammon's NLM and (b) Visor algorithm 
and Visor algorithm. It is important to note that S 
VISOR algorithm in terms of accuracy in mapping. 

Sammon's NLM are seen to be widely distributed, 
algorithm are found to assemble togethe 


ammon's NLM is able to outperform 
The mapped data obtained using the 
whereas those determined by the VISOR 


г. Thus, a particular Point and its пе 
are f eighbors cal 
easily be identified from the mapped data yielded by Sammon’s NLM whee VISOR 


algorithm cannot offer a good visibility of the Mapped data. Moreover VISOR algorithm Ë 
found to be computationally faster than the Sammon's NLM. Interested add EE rele 
to Dutta and Pratihar 49] for a detailed comparison of different. mapping methods ' 
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4.3.3 Working Principle of the VIGA 


"~ 


Let us suppose that an objective function involving L dimensions (where L > 3) is {о 
be optimized using a СА. In such a case, we cannot visualize search direction оп the 
surface of objective function while doing optimization and the GA finds the optimal solution 
after conducting its search through a few iterations. We may not even know the type of 
processing actually takes place inside the GA, although we get the optimal solution. Thus, 
a GA works almost like a black box. In a VIGA, an attempt will be made to collect 
topological information of the higher dimensional space by mapping a set of data into a 
| lower dimensional space (2 or 3-D) using some methods, namely Sammon's NLM, VISOR, 
SOM, and others. As the neighboring points in a higher dimensional space try to keep 
i their positional information and topological relation intact in a lower dimensional space 
during mapping, it is possible to locate the global basin in a higher dimensional space 
by examining the data points in 2 — D or 3 — D. A human being has a capability to 
p study the data points in 2 — D or 3 — D and thus, to identify the probable location of the 
global optimum. This information regarding a possible region of global optimum in a lower 
dimensional space is transformed into a higher dimensional space using a reverse mapping, 
that is generally implemented with the help of a look- up table. Figure 4.9 shows a schematic 
diagram explaining the working principle of the VIGA. The GA is being informed by the 
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Figure 4.9: A schematic diagram to show the working principle of a VIGA. Ay 


ossible location of good solutions at each iteration. As good solutions are added 
ation, the convergence rate of the VIGA is expected to be higher thi - 
3A. The VIGA is seen to be almost. five times faster tha 
'er, the performance of the VIGA is found to be func 
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4.4 Scheduling GA *] 
Scheduling problems belong to а special class of optimization Appi E T f 
the position of different elements but also their adjacency and or re ~ ure 
us take an example of a Travelling Salesman Problem (TSP) dois vs die type | 
of scheduling problem), where a salesman has to visit all n cities (each МА die ed only 
once) by travelling either through a minimum distance path or in minnu рле Or at 4 
minimum cost. Let us assume that all n cities are connected to each other by some suitable 
paths. Let us also consider this problem to be a symmetrical one, in which the travelling 
distance/time/cost between any two cities is independent of the direction of travel. Fig. 410 


shows a symmetrical TSP. 








Figure 4.10: A symmetrical TSP, in which all the cities are connected to each other in both ways, 
The symbol d;; represents the Euclidean distance between the cities i and j. Т 


tete he distance 
matrix is shown below. 


di diz di dj dis dig о 50 45 40 70 To | 
| da dos Фа doy dos dag 90 0 40 60 60 50 
| dı d3 зз 4 das dag _ | 45 40 0 40 35 40 
| dà de) Фаз Фа das de | |40 60 40 0 45 70 
ў ds; dz? 5з Чы dss dsg 70 60 35 45 0 40 
dai dẹ? des Фа des deg 75 50 40 70 40 0 
| 


| With these assumptions, the above scheduling problem can b 
tion problem. If a salesman starts from a particular city, he wi 
routes/sequences through which he can | 
optimal path out of all (n — 1)! 











| posed as an optimiz® | 
have (n — 1)! possible _ 
touch all the cities once. He will have | а: he. 












a possibilities. As the salesman has to visit all he 
otn posible seater pendent of selection of the starting city. "Thus, he Pagers 
Nu ERE Sequences and the optimal one is to be determined, — : ij per 
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4 
E 
Tis s 1 15 c 
[he distance mat 
(ly i i | | 0 Z 
d» | i | г А 1 hl) Gil) E) 
| ila r (1 |) 15 |) 
| М1 . | || | Ti 
| бы if 0 40 
dei 15 I" 
| 15 | lea from the а i that a Simple GA (об Ау mav not be suitable to solve 
the scheduling problen ntronal CI er operators (such as single-point, two- 
point multi-point unifort леја me infeasible children solutions. It is 
important to note that mutati ised to solve the above problem, unless it 
is required in a spec! il situation. Thus, a special type of crossover ор rator is to be used to 
. Я - i mE | CLA | | | | : 
determine the optimal schedule. Some о: these зремаке d crossover operators are discussed 


below in detail [50 


4.4.1 Edge Recombination 
+ al. [51], in which an importance is given on adjacency 


It was developed by Whitley € | 5 f 
| position of the elements. We maintain an edge 


information but not on the order and | | | m in a 
table, which carries information of each element and its possible links. Thus, it gives the 
“= : LE eS L ik 
connectivity information of each element 
Let us consider a travelling salt | 
Let us also assume that children solutions 
the two parents given below. 


STIL ATI problem involving nine cities (refer to Fig. 4.12). 
are to be created using edge recombination from 


ыл 


7 8 9 
2ч 


æ 
==. 
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bo 
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Pri: | 
5 Pr: 9 


A Fig. 4.12 shows the edge table constructed for the said purpose. 
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Connectivity information/Links 


2,944 
V, 2, 8,6 
2,4, 9,4 
JA, 5,4 
if, 6, 8 
5,7,2 
6, 8, 9 
7,9, 5,2 
B. A, TA 


1 
P, ~ ar М 


T. м ۹ 


y E 
fl 24 == * 


Figure 4.12: Edge recombination operator 


The following procedure is adopted to determine a child solution, say child 1, that is, 


pA 


We continue this process, until the tour is completed 
resulting child 1 will be as follows: 





Let us start the child tour with the starting city of Prl, that is, 1, 


Аз city 1 has been selected, all occurrences of 1 are to be deleted from the right-hand 
side of edge table. 


City 1 is connected to the cities 2, 3, 4 and 9. Now, the cities 2. 3 and 9 have three 
remaining connectivities each in the edge table, whereas city 4 has only two remaining 
links. Thus, 4 is selected as the next city to city 1. 


We eliminate all entries of city 4 from the right-hand side of edge table 


City 4 has the links to the cities 3 and 5 ( 
the cities 3 and 5 have two remaining 
the cities 3 and 5 may be selected at 
city 4. 


› {as 1 has already been selected). Now, both 
links in the edge table, Thus, any one out of 
random. Let us select city 3 as the next city to 


We remove all entries of city 3 from the right-hand side of edge table 


after touching all the cities once. 


I od (а ШАА, 
ae tae Ad) = 
ME ГА, 2 
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ee 
+ „Пагу, child 2, that is, Ch2 ma Ec 
d 2) as the starti i SUE ned consideri i i 

city of PFA arting city of Cho. Using nsidering city 9 (vhat is, starting 


gch 


t CE 
he above steps, Ch2 is obtained like the 


J $4.0 5.8 20 E 


following: 
Cha: 


4.4.2 Order Crossover #1 


Order А #1 was proposed by Davis [92], in which a part of hild soluti 
„actly copied from a part UN T one child solution is 
E ect = | ; 3 poe of one parent and the other part of that child inherits the Пт 
of remaining е 106 nts of other parent. А care has to be taken, such that | | al dy 
selected in the child solution should not Me j| an element already 


E е) T 
to create two child еп: 


The steps to be followed to get two children solutions from the above two parents are: 
e Select two crossover sites at random, as shown above. 


e To determine child 1, that is, Chl, the elements — 1, 2, 9, 8 (lying between the two 
crossover sites) are directly copied from Pr2, keeping their locations and order intact. 


e Child 1 tour then begins from Frl, starting from the first position after the second 
crossover site and searching towards the initial elements of Frl. 


e Cities (elements) 8, 9, 1, 2 are already present in child 1 solution, which should not 


be considered again. 


e The 8 — th, 9 — th, 1— st, 2— nd and 3 — rd positions of Chl are selected as follows: 


Chl[8] = Pr1|3] = 3, 
Сп] = Pri(4] = 4, 
Chil] = Pr1[5] = 5, 
Chi(2) = Рг1[6) = 6, 
Chi(3] = Pri[7] =T. 


Thus, the obtained Chi will look as follows: 
(за. 122038, ШЧ 


ned and it will look like the following: AR 


ЬУ 


ur U 
"s E ps are À 
> 4 
Um st 4 
LJ LS + 
NM > 


см: 5 6 7 


Ch2 can be determi 


Similarly, child 2, that is, 
cement, ЫЕ 
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4.4.3 Order Crossover #2 


Order crossover #2 was developed by Syswerda |5; VE 

at random and the order in which the elements of these positions 

imposed on other parent to create the children solutions 
Let us suppose that the following two parents participate 


+l. in which some key positions are selecte 


appear їп one parent. [s 


in order crossover ie 


Pr] а 34 5 15. 7:0 9 
= ж: + * 
Pro: 9 4.5 12 9 6 7 6 


The principle of this operator is explained below ШІ sl ea 
e Select the positions 2 4. Тапа 8 as the key positions, а! random. 


e The elements of Pr2 at the above mentioned key positions are 4, 1, 8, 7. To determing 


z | | f t] | 1 |; 7 у : т А | } 
Chl, the ordering Of these elements 1. 1. 5, Т1П Рта 15 ппроѕ‹ don Pri 

| Г] In Prl. the elements |. 1,5 апа T ari found in positions i th. | "e T 5 
(— th, respectively 


• In Chl, the elements in these positions (that is, ] st, 4 th, 1 th and 8 — th) 


are selected by matching the order of the elements 4, 1, 8, 7 as found in Pr2, that is, 


Сп] = 4; Сп) = 1; Ch1[7] = 8; Ch118 
e The remaining elements of Chl are directly copied from Pr] 
Thus, Chl will look like the following: 
Chl Као ok 5 5 8 
Similarly, Ch2 can be determined, which will look as follows: 


Che: 3 2: 5 4 1'9 T7 ® є 
4.4.4 Cycle Crossover 


are preserved while determining a child solution. 
Let us consider two parents, as shown below, w 
to create two children solutions. 


lements of a parent 


hich will participate i 
ch will participate in a cycle crossovet 


Eri 1523. 4'516-7'8 9 
* 
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The mechanism of cycle crossover is explained with the help of the following steps: 


d iens Chl, select a parent (say Pr1) and the starting position of the cycle (say 
Prii3j - 3). at random. Thus, the 3 — rd element of Chl is nothing but element 3, 
that is, C'A1|3] = 3. 


ees then searched to check the presence of element 3 and it has been found in the 
1 —st position. The first element of Ch] is selected from the first element of Prl, that 
is, Ch1[1] = Рт] 2 1 


• Pr2 is again searched for a presence of element 1 and it has occurred at the 4 th 
position. Thus, the 4 — th element of Рт] has been copied as the 4 — th element of 
Chl, that is, САЦА Pr1lj4| = 4. Similarly, we determine 


CR1|2] = Pri[2] = 2, 
CA] Р] Pr1i5 5 


l'his completes one cycle because element 5 is seen to be present at the 3— rd position 
of Pr2 and the corresponding 3 — rd position element of Рг1 is element 3, which has 


already been selected as the starting element of the cycle 


e The remaining elements of Chl are selected directly from Pr2, as follows: 


Chil6] = Pr2(6] = 9, 
САТ = Pr2[7] = 8, 
СЋИВЈ = Pr2[8] = т, 
Стад) = Pr2[9| = 6 


Thus. Chl is found to be like the following: 


Chl ] :2  @7b :# X [ D 


Using the same procedure, Ch2 can be determined as follows: 


Grow 3 + 652890 tft 5 9 


4.4.5 Position-Based Crossover 
| Position-based crossover was int roduced by Syswerda [53], in which an attempt is made to 
preserve position information of different parent elements in the child solution. 
Let us consider the following two parents, which will participate in the position-based 
Crossover: ғ 










Рејно 1 2.304 об 8:19 


* * * + 


Pr2: 3 4 512 9 8 7 6 
"he rinciple of position-based crossover is discussed with the help of following steps: 


== 
= 
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r points (say 1-st, 4-th, 7-th, 9-th) on 
ly copied from Prl (by keeping 
set Chi[] = 1, Calla] = 4, 


e a number of crossove 
-1, 4, 7, 9 are direct 
t) to Chl. Thus, we 


e To determine Chl, choos 
a parent, say Prl. The elements 
their position information шас 
СВ] = 7, Ch1[9] = 9. 

e The remaining elements of Chl are selected {гош Pr2 as follows: 

см] = Рг2[1] = $; | "sn 

Ch1[3| = Pr2(3) = 5, as Pr2[2] — 4 has already been included in Ch Ly 

Ch1[5] = Pr2|5] = 2, as Рг2|4| = 1 has already been selected as С All| || element; 

Chl[6| = Pr2|T| = 8, as Pr2|6| = 9 has already been considered аз СУ 


САВ] = Pr2[9| = 6, as Pr2l8| = T has already occurred as СИТ). 


РЕ " 


) Thus, the resulting Chl solution will look as follow 


Similarly, Ch2 can be determined, which wil: | ће follow! 
Ch? 2 4 " R Ü I 


4.4.6 Partially Mapped Crossover (PMA) 
It was proposed by G ldberg and wle hint | i : 
| proposed Û] че гр and Lingi 1 which a pal 1 а parent solution lying 
between two crossover = 25 15 directiv coped into i hl ae "Y, mS 
oet | | pied into a child solution. Thus, the position, 
adjacency and order oi that part ol the parent 3 req 0 int Fin FI Lt) 4 ; 
| \ hat р І пі remain intact in the child solution. 
Let us take an example of the following two parents, wil h will nar! { : 
: теп hich will participate in a Partially 


Mapped Crossover (PMA | 


Ea] г» | F Р А = x 
P: | e || 2 d | и | = | Р Г Ы ü 
Mes. " - - РЧ : > - 
Pre: 3 4 0 1..2 9 .& 7 Б 


The steps involved in the PMX are: 


A e Select two crossover sites at random, as shown above 
e The elements = 4, 5, 6, 7 of Pri (lyi Wa 
copied into Chi Tu à 7 of Prl (lying within the two crossover sites) are directly 
| to Chl. Thus, we get СМА] = 4, Ch1[5] = 5, Ch116] s a д ae 
ТАН = 6, and CRIT = T. 


e The remaining elements of C 
nts of Chl are determined as f 
| So гашен ietermined as follows: The sear "c 
nee ith residing in between the two crossover sit Б з search абын the 
at the 2 — nd position of Pr2, that is Pr2(2] an The element РАНА =, 
ا‎ [ 2 |. | us, the 


Chl will be filled up by an element of Pr2 located at 4 4 – nd poss 


— th position, that is, city L 


Ста] = Pr2ja) = 1. 
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similarly, th i 
Sin y, the element Pr1[5| = 5 is seen to be present at the 3 — rd position of Pr2, 


that is, Рг2[3]. Th Ет i. 
Pr? located | | T us, the 3 — rd position of Chl will be occupied by an element of 
| at the 5 — th position, that is, city 2. f 


Ch1[3| = Pr2[5] = 2. 


D arl 7 fé - al А OD А н z 
Similarly, we can determine the following elements of Chl: 


Ch1[89] = Pr2[6] = 9, 
САВ] = Pr2[7] = 8. 


a ~ гет ёч - B | ^ = JF "2L : | / "E 5 e Е ; n = al 
The папе ELement ol GAL is direc Liv copied trom Рта. toat 18, C Ali = Pr2 


3. у L*] L^ 


Thus. Chl is obtained as follows 
Ch2 can be determined following the similar pro« edure and it will look like the following: 


Note: Partially mapped crossover may sometimes give rise to à child solution, in which a 
particular city may occur ше than once and some other cities might also be missing from 
it. 


Let us take an example, in which two parents ars participating in the PMX, as shown 
below. 
i y 1 D T ~ Hd 
p 1 | | j t| " T | 


С] Р) | К. і Ü 0 o T d 
Dom cU sl 19) 7 358 


In Chl. city 5 has оссш red twice and city 2 is missing trom It The cities lying inside the 

first brackets are generally not disturbed. Thus, the first element of Chi, that is, city 9 is 

AL nonu c К xi ; FI ' Ci - =i 

to be replaced by the missing city, that is, 2. The modified Chl may be written as follows: 
' E 2 ^ LG De 6 " 


су 201 9 4 85 6 © 7 9 


C ~ а мо ia found to be like the following: 
Similarly, the modified Ch2 is found to be like the E 


Ch2: 4 3 5 1 S 9 7 8 6 


es of different crossover operators are 


It is important to mention that the performanc 
problem-dependent [50]. 
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4.5 Summary 

| follows: 
been summarized as follows | е0 | 
sover and mutation operators used in the real-coded GAs 
the help of suitable numerical examples, 
; which elitism has been used. Ап 
ed Interactive GA, where some 


The content of this chapter has 

The mechanisms of different cros | 
bv various investigators, have been explained with | 
The working principle of a micro- GA has been discussed, ш 
introduction is given to another faster СА named Ма sa Ee el со 
mapping methods like Sammon’s NLM, VISOR algorithm, anc о 8 A Е инок | 
data from a higher dimensional space to а lower dimensional space or : 4 > | па 
scheduling problem, not only the position of different elements put aiso t en d es and 
order are to be considered. А number of crossover operators have been proposed by various 
researchers to solve the said problem The principles of some of these operators have been 
explained with the help of some appropriate example 


4.6 Exercise 
1. When and why do we preter à real-coded GA to a binary coded GA ? 
2. Define elitism and briefly explain the principle oi micro- GA 
3. Can the VIGA be taster than the SGA ! Explain 11 


4 Why do we need a Spec ial type oi crossover opel ator tor solving a scheduling problem? 
Explain briefly different crossover operators used in the scheduling GA. 


5. To solve an optimization problem using a real-coded GA, let us assume that a mating 


pair (consisting of two solutions) is found to be as follows: 


Рт] 16.85, Pr2 = 19.50 


Determine the children solutions using different crossover operators given below 


(a) Linear crossover, 
(b) Blend crossover with a — 
=O 


(c) Simulated Binary Crossover (SBX) assuming the probabil 
contracting and expanding zones as follows: 


0.6, that is, BLX — 0.6, assume the random number 


ity distributions for the 


| C(a) = 0.5(q + 1)a*, 
Ет(а) = 0.5(q + 1) ерту, 


where а is the spread factor and q = 4. Assume the random number r — 0.7 


6. To solve an optimization problem utilizing a real-code 
value corresponding to an original solution Pr 
conditions: 


d GA, determine the mutated 
original = 19.68 under the following 
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(a) Use random mutation, assuming random number r = 0.6 and the maximum 
perturbation A = 2.0 


(b) Use polynomial mutation, assuming the random number т = 0.6. 


7. Develop the computer programs for Sammon's NLM and VISOR algorithm. Map the 
4-D data shown in Table 4.1 into 2-D using the above methods. 


Table 4 | А Set ot 21) x 4 lata 
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8. Let us consider a TSP problem involving eight CIMES A, 12 DE 2, Н. А 


heduli СА with various types 01 t rossover operator 15 LO be used to solve the said 
scheduling GA with val Ly] | i a id 
ti tization problem Determine children solutions of the following two parents: 
optimiza yrobiet 
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(a) Use edge recombination. The edge table is shown in Fig. 4.13. 


).nd and 5-th sites as the crossover sites. 


(b) Order crossover #1, assuming 4 
(c) Order crossover #9, considering 3-rd, 
(d) Cycle crossover assum 
(e) Position-Based Crossover conside 


(f) PMX assuming 2-nd 


4-th and 7-th as the key positions. 
üng 4-th as the starting position. | 
ring 2-nd, 4-th and 6-th as the crossover points. 


and 5-th sites as the crossover sites. 
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H:A. B, E, G 


Figure 4.13: Edge table. 


:: The positions are counted from the left side. 
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